Power operation of intelligent devices

ABSTRACT

A voice controlled device comprises a plurality of microphones, a communication interface, and a controller. The plurality of microphones are configured to receive voice inputs from one or more users for operation of the voice controlled device. The communication interface is configured to receive sensor data from an external device, the sensor data describing an environment of the voice controlled device. The controller is configured to operate a first subset of the plurality of microphones in response to first sensor data and a second subset of the plurality of microphones in response to second sensor data.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of and priority to U.S. ProvisionalApplication No. 62/559,199 filed Sep. 15, 2017, which is herebyincorporated by reference in its entirety.

FIELD

This disclosure relates to the field of kitchen devices or appliancesand bathroom devices or appliances, and more specifically, intelligentkitchen devices and intelligent bathroom devices that provide functionsto a user in response to sensor data collected in relation to the user.This disclosure further relates to intelligent kitchen devices andintelligent bathroom devices that are connected to each other and/oranother network through a home hub communication device and may becontrollable via a mobile application and/or voice commands.

BACKGROUND

A virtual assistant provides information, performs tasks, or providesservices to a user. The virtual assistant may be a device or a softwareagent. The virtual assistant may receive voice commands through naturallanguage processing. The information provided by the virtual assistantmay include data that is easily accessed from the internet, includingthe weather, traffic, music lyrics, or the news. The tasks performed bythe virtual assistant may include making dinner reservations, requestinga driver, performing an internet search, or checking the news. Theservices provided by the virtual assistant may include playing music,setting a thermostat, reading a book, setting an alarm, or placing acall.

SUMMARY

One implementation of the present disclosure is a system of householdappliances for providing services according to user identity. The systemincludes a first appliance, a server, and a second appliance. The firstappliance is configured to collect sensor data associated with anidentity of a user. The server is configured to receive the sensor datafrom the first appliance and analyze the sensor data to determine theidentity of the user. The second appliance is configured to provide adevice function to the user based on the identity determined from thesensor data collected by the first appliance.

Another implementation of the present disclosure is a method forproviding services with household appliances according to user identity.The method includes receiving sensor data collected by a firstappliance. The sensor data is associated with an identity of a user. Themethod includes analyzing the sensor data from the first appliance todetermine the identity of the user, accessing a user database using theidentity of the user for a user configuration, and generating a commandfor a device function for a second appliance based on user configurationfor the identity of the user.

Another implementation of the present disclosure is a communicationsystem for providing feedback data for at least one water consumingdevice. The communication system includes a data collection interface, acontroller, and an output interface. The data collection interface isconfigured to receive user data from at least one collection device. Thecontroller is configured to perform an analysis of the user data fromthe at least one collection device. The output interface is configuredto provide feedback data based on the analysis of the user data to awater consuming device.

Another implementation of the present disclosure is communication systemfor providing feedback data for water consuming devices. Thecommunication interface includes a data collection interface, acontroller, and an output interface. The data collection interface isconfigured to receive user data from at least one water consumingdevice. The controller is configured to perform an analysis of the userdata from the at least one water consuming device. The output interfaceis configured to provide feedback data based on the analysis of the userdata.

Another implementation of the present disclosure is method forcoordination of household devices. The method includes receiving, at adata collection interface, sensor data from at least one first device.The sensor data describes a state of a user. The method includesperforming, using a controller, an analysis of the sensor data from theat least one first device and generating, using the controller, feedbackdata based on the analysis of the sensor data. The feedback data isconfigured to operate a second device based on the sensor data from theat least one first device. The at least one first device or the seconddevice is a water consuming device.

Another implementation of the present disclosure is an apparatus foraggregation of water condition at household appliance. The apparatusincludes a communication interface and a controller. The communicationinterface is configured to send a reporting message indicative of datacollected by at least one appliance to a central server. The controlleris configured to receive an analysis message from the central server.The analysis message indicates a condition of water from the at leastone appliance or a condition of water in a geographic area associatedwith the at least one appliance. The controller is configured to providean alert in response to the analysis message.

Another implementation of the present disclosure is a method foranalyzing a reporting a condition of water. The method includesreceiving a reporting message indicative of sensor data collected by atleast one appliance. The reporting message includes an identifier forthe at least one appliance. The method includes determining anindication of a condition of water based on the sensor data, accessing adatabase according to the identifier for the at least one appliance todetermine one or more neighboring users, generating an analysis messageincluding the indication of the condition of water, and sending theanalysis message to the one or more neighboring users.

Another implementation of the present disclosure is an apparatusincluding at least one sensor and a controller. The at least one sensoris configured to collect data for a water condition associated with atleast one first appliance. The controller is configured to perform ananalysis of the data for the water condition, identify a resultantinstruction in response to the analysis, and send the resultantinstruction to at least one second appliance.

Another implementation of the present disclosure is an apparatusincluding a light source array, a communication interface, and acontroller. The light source array includes one or more directionallight sources. The communication interface is configured to receive userdata from at least one appliance. The user data indicates an identity ofa user. The controller is configured to analyze the identity of the userand activate the one or more directional light sources in response tothe identity of the user.

Another implementation of the present disclosure is a method forcommanding a light source array. The method includes receiving user datafrom at least one appliance. The user data indicates an identity of auser. The method includes analyzing, by a processor, the user data toidentify at least one appliance to be illuminated for the user. Themethod includes generating, at the processor, a light command for alight source array comprising one or more directional light sources toilluminate at least a portion of the at least one appliance to beilluminated.

Another implementation of the present disclosure is an apparatusincluding a communication interface and a controller. The communicationinterface is configured to receive user data from at least oneappliance. The user data indicates an identity of a user. The controlleris configured to analyze the user data to identify at least oneappliance to be illuminated for the user. The controller is configuredto generate a light command for a light source array including one ormore directional light sources to illuminate at least a portion of theat least one appliance to be illuminated.

Another implementation of the present disclosure is a mirror assemblyincluding a mirror substrate, a sensor, and a controller. The mirrorsubstrate is configured to provide a reflection of objects including auser. The sensor is configured to collect sensor data from the user. Thecontroller is configured to analyze the sensor data and select anauxiliary command for an auxiliary device coordinated with the mirrorassembly. The auxiliary command is selected for the user.

Another implementation of the present disclosure is a method forcoordinating at least one auxiliary device with a mirror assembly. Themethod includes receiving sensor data for a user at the mirror assembly,performing an analysis on the sensor data, and selecting, based on theanalysis, an auxiliary command for an auxiliary device coordinated withthe mirror assembly.

The auxiliary device is spaced apart from the mirror assembly and theauxiliary command is selected for the user. Another implementation ofthe present disclosure is an apparatus including a controller for amirror assembly and a communication interface. The controller isconfigured to analyze sensor data received from a sensor and select anauxiliary command for an auxiliary device coordinated with the mirrorassembly. The auxiliary command is selected for the user. Thecommunication interface is configured to send the auxiliary command tothe auxiliary device.

Another implementation of the present disclosure is an apparatusincluding a voice controlled device includes an array of microphones, acommunication interface, and a controller. The array of microphones isconfigured to receive voice inputs from one or more users for theoperation of the voice controlled device. The communication interface isconfigured to receive sensor data from an external appliance, and thesensor data describes an environment of the voice controlled device. Thecontroller is configured to operate a first subset of the array ofmicrophones in response to first sensor data and a second subset of theplurality of microphones in response to second sensor data.

Another implementation of the present disclosure is an apparatusincluding a voice controlled device includes a housing, a dock, amicrophone, and a coupling mechanism. The dock is configured to connectthe housing to multiple host appliances. The coupling mechanismconfigured to receive an identification value indicative of dockingbetween the voice controlled device and a currently connected hostappliance. The microphone is configured to receive one or more voiceinputs for the currently connected host appliance, and a command isprovided based on the one or more voice inputs and the identificationvalue.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments are described herein with reference to thefollowing drawings, according to an exemplary embodiment.

FIG. 1 illustrates a bathroom setting that includes multiple appliancesor intelligent bathroom devices connected through at least one home hubcommunication device, according to an exemplary embodiment.

FIG. 2 illustrates a kitchen setting that includes multiple appliancesor intelligent kitchen devices connected through at least one home hubcommunication device, according to an exemplary embodiment.

FIG. 3 illustrates a communication network for the sets of appliances ofFIG. 1 and/or FIG. 2, according to an exemplary embodiment.

FIG. 4 illustrates a flow of data between appliances for providingservices to a user, according to an exemplary embodiment.

FIG. 5 is a flow chart for operating at least one water consumingdevice, according to an exemplary embodiment.

FIG. 6 illustrates a flow of data between appliances for controlling afirst device based on data collected by a second device, according to anexemplary embodiment.

FIG. 7 illustrates bidirectional flow of data between appliances,according to an exemplary embodiment.

FIG. 8 illustrates another bidirectional flow of data betweenappliances, according to an exemplary embodiment.

FIG. 9 illustrates a control system for the appliances or home hubcommunication device, according to an exemplary embodiment.

FIG. 10 is a flow chart for the home hub communication device, accordingto an exemplary embodiment.

FIGS. 11A and 11B illustrate a mirror and cabinet including a home hubcommunication device and user interface, according to an exemplaryembodiment.

FIG. 12 illustrates another user interface which can be presented via amirror and cabinet, according to an exemplary embodiment.

FIG. 13 illustrates an example transitional user interface and mirror,according to an exemplary embodiment.

FIGS. 14A and 14B illustrate a housing for a communication module andmirror interface, according to an exemplary embodiment.

FIG. 15 illustrates controls for the communication module of FIGS. 14Aand 14B, according to an exemplary embodiment.

FIG. 16 illustrates a panel for the communication module of FIGS. 14Aand 14B, according to an exemplary embodiment.

FIG. 17 illustrates a toilet including a home hub communication deviceand user interface, according to an exemplary embodiment.

FIGS. 18A and 18B illustrate a toilet and a home hub communicationdevice including multiple automated devices, according to an exemplaryembodiment.

FIGS. 19A and 19B illustrate an automatic toilet seat, according to anexemplary embodiment.

FIG. 20 illustrates a control panel for the home hub communicationdevices, according to an exemplary embodiment.

FIG. 21 illustrates a kitchen appliance including a home hubcommunication device, according to an exemplary embodiment.

FIG. 22 illustrates a shower device including a home hub communicationdevice, according to an exemplary embodiment.

FIG. 23 illustrates a communication system for the home hubcommunication device FIG. 22, according to an exemplary embodiment.

FIG. 24 illustrates a bathtub including a home hub communication device,according to an exemplary embodiment.

FIGS. 25 and 26 illustrate an interchangeable speaker.

FIG. 27 illustrates a bathroom setting that includes multiple appliancesor intelligent bathroom devices compatible with the interchangeablespeaker.

FIGS. 28A, 28B and 28C illustrate a docking detection system for theinterchangeable speaker.

FIGS. 29A and 29B illustrate an inductive coupling system for theinterchangeable speaker.

FIGS. 30A, 30B, AND 30C illustrate a magnetic coupling system for theinterchangeable speaker.

FIG. 31 illustrates a flow chart for the interchangeable speaker.

FIG. 32 illustrates a flow chart for sound filtering based onintelligent bathroom devices.

FIG. 33 illustrates a microphone array for a speaker.

FIG. 34 illustrates a microphone array across multiple appliances.

FIG. 35 illustrates a fan including a microphone array.

FIG. 36 illustrates a flow chart for operation of the microphone array.

FIG. 37 illustrates a remote charging and detection system.

DETAILED DESCRIPTION

A virtual assistant may record voice commands using a microphone andsend the voice commands to a cloud system through a network connection.The cloud system may collect information in response to the voicecommands and reply to the virtual assistant with the information. Thefollowing embodiments include household appliances that are integratedwith a virtual assistant or otherwise interact with a virtual assistantto share data among the household appliances. The data is collected byone appliance and utilized by another appliance. The utilizing applianceprovides services to a user, and the services are customized based onthe data shared from the collecting appliance. In one example, theshared data includes identifying information for the user orcharacteristics of the user that are used to apply customization,configuration, or preferences to the services provided by the utilizingappliance. The following embodiments improve the operation of thehousehold appliances by increasing the data set from which personalizeddevice functions can be selected and performed. For example, dataotherwise unavailable at the utilizing appliance is detected at sharedfrom the collecting appliance. The following embodiments improve thecommunication networks by standardizing the exchange of data amonghousehold appliances.

Summary of Appliances

FIG. 1 illustrates a bathroom setting 10 that includes multipleappliances or intelligent bathroom devices connected through at leastone home hub communication device, according to an exemplary embodiment.The intelligent bathroom devices may include one or more of anintelligent mirror 1, a programmable shower 2, a bathtub sensory device3, a bathtub level device including a drain 4 and faucet 5, anintelligent toilet 6, a toilet seat 7, a sink faucet 8, and light guidesincluding a light source 9 a and projected guides 9 b. Each of theintelligent bathroom devices is configured to collect data indicative ofa user, communicate the data indicative of the user to anotherintelligent bathroom device either directly or indirectly, and provideservices to user based on data indicative of the user that is receivedfrom other intelligent bathroom devices.

The intelligent mirror 1 may include a screen or other user interface toprovide selections to any of the intelligent bathroom devices or otherdevices. For example, commands received at the intelligent mirror 1 maybe transmitted to any of the other devices (e.g., to turn on, turn off,start a function, or make a selection). Further, the status or othersettings of the other devices may be displayed by the intelligent mirror1.

The programmable shower 2 may provide a shower sequence to the user. Theshower sequence may include any combination of a sequence ofpredetermined intensities for a shower spray, a predetermined sequenceof different shower heads, a set of temperatures, or other patterns. Theshower sequences may be selected based on an identification of the useras facilitated by another intelligent bathroom device or other devices,data collected by the other devices, or a user selection may through theother devices.

The bathtub sensory device 3 may provide a predetermined (e.g., userdefined) pattern of any combination of sound, hydrotherapy, aroma,lighting, and massage. The bathtub sensory device 3 may include speakersintegrated with or embedded in the bathtub. The predetermined patternmay be generated or selected based on the identification of the user asfacilitated by another intelligent bathroom device or other devices,data collected by the other devices, or a user selection may through theother devices. The status or pattern of the bathtub sensory device 3 maybe transmitted to and displayed by the intelligent mirror 1 or otherdevices.

The bathtub level device, which may include an automated drain 4 andautomated faucet 5, may automatically regulate the fill level of thebathtub. The bathtub level device may maintain a predetermined level ordepth of the water in the bathtub and maintain a predeterminedtemperature or range of temperatures of the water. The bathtubautomatically monitors changes to water temperature in the bath andadjusts the water temperature coming out from the faucet 5, or otherlocation accordingly. For example, the bathtub can adjust the watertemperature coming out of the faucet 5 based on remaining volume thatneed to be filled to achieve the correct final desired state based ondifferent ambient temperatures. The bathtub predetermined level andpredetermined temperatures can be changed to new set values in whichcase the bathtub dynamically changes its status to achieve the new setvalues. For maintaining water temperature once the bath is ready (i.e.,when the water level is already at its target fill level), a calculatedvolume of water is drained from the bathtub and added to the bathtub tomaintain the temperature as the water is affected by the ambientenvironment. The fill level that is maintained by the bathtub leveldevice may be based on an identification of the user as facilitated byanother intelligent bathroom devices or other device, data collected bythe other devices, or a user selection may through the other devices.The change in water level as detected by the level device can be usedfor identification of the user and can prompt other changes to thelighting 9, music 1, and hydrotherapy accordingly. The status or patternof the bathtub level device may be transmitted to and displayed by theintelligent mirror 1 or other devices.

The intelligent toilet 6 may provide automatic opening and closing aswell as flushing and other sanitation features. The intelligent toilet 6may provide lid and ring orientations or positions based on anidentification of the user as facilitated by another intelligentbathroom device or other devices, data collected by the other devices,or a user selection may through the other devices. The status of theintelligent toilet 6 may be transmitted to and displayed by theintelligent mirror 1 or other devices.

The toilet seat 7 may include a heater that increases the temperature ofthe toilet seat 7 for comfort of the user. The temperature may beselected based on an identification of the user as facilitated byanother intelligent bathroom device or other devices, data collected bythe other devices, or a user selection may through the other devices.The toilet seat 7 may collect biometric data or biologicalcharacteristics from the user. The status of the toilet seat 7 orconclusions drawn from the data collected at the toilet seat 7 may betransmitted to and displayed by the intelligent mirror 1 or otherdevices.

The sink faucet 8 may provide a predetermined volume of water, apredetermined temperature of water, or a predetermined flow velocity ofwater based on an identification of the user as facilitated by anotherintelligent bathroom device or other devices, data collected by theother devices, or a user selection may through the other devices. Thesink faucet 8 may collect data on flow quality, filtration dataincluding particulate data (e.g., mineral level or metal level), ormicro-organism data. The status of the sink faucet 8 may be transmittedto and displayed by the intelligent mirror 1 or other devices.

The light guides, including light source 9 a and projected guides 9 b,may project light on one or more other devices to guide the user to thedevice or help illuminate the area near the device. In response to afunction being selected at one of the devices, user selection data, or alocation and/or direction associated with a user selection, istransmitted to the light source 9 a to define a direction for projectingthe projected light guides 9 b. The status of the light guides may betransmitted to and displayed by the intelligent mirror 1. In onealternative, the light guides are implemented as versatile tiles or techtiles. The versatile tiles may include lights, speakers, sensors, and/orheaters implemented as flooring.

FIG. 2 illustrates a kitchen setting 20 that includes multipleappliances or intelligent kitchen devices connected through at least onehome hub communication device, according to an exemplary embodiment. Theintelligent kitchen devices may include one or more of a kitchen faucet21, a dishwasher 22, a garbage disposal 23, a refrigerator 24, a waterheater 25, and a water filter 26. Each of the intelligent kitchendevices may be connected to a water supply 27 and a drain 28. Each ofthe intelligent kitchen devices is configured to collect data indicativeof a user, communicate the data indicative of the user to anotherintelligent kitchen device either directly or indirectly, and provideservices to user based on data indicative of the user that is receivedfrom other intelligent kitchen devices.

A control center for the kitchen may be implemented using a displayscreen. The display screen may be incorporated in one or more of theintelligent kitchen devices. For example, the refrigerator 24 mayinclude a control center with a controller and a display 16. In additionor in the alternative, any of the intelligent kitchen devices mayinclude a similar control center display 16. The control center display16 may communicate with any of the intelligent kitchen devices toreceive commands or data or control the other intelligent kitchendevices.

The kitchen faucet 21 may provide a predetermined volume of water, apredetermined temperature of water, or a predetermined flow velocity ofwater based on an identification of the user as facilitated by anotherintelligent bathroom device, kitchen device, or other devices, datacollected by the other devices, or a user selection may through theother devices. The status of the kitchen faucet 21 may be transmitted toand displayed by the control center display 16. The kitchen faucet 21may include a flow sensor configured to measure a flow of water to thekitchen faucet 21, which is transmitted to the control centercontroller.

The dishwasher 22 may provide various washing cycles for washing dishes.The washing cycle may include a predetermined pattern of washing cycles,a predetermined type of washing pattern (e.g., gentle cycle, pots andpans, sanitize, extra heat, steam), a temperature setting, a turbiditysetting, a water volume setting, or another setting. The status of thedishwasher 22 may be transmitted to and displayed by the control centerdisplay 16. The dishwasher 22 may include a flow sensor configured tomeasure a flow of water to the dishwasher 22, which is transmitted tothe control center controller.

The garbage disposal 23 may include a garburator or cutting blades forcutting food wastes or other material into small pieces for passingthrough plumbing. The garbage disposal 23 may be mounted in line betweenthe drain of the sink and the trap of the plumbing. The garbage disposal23 may include a timing setting for an amount of time for the garbagedisposal 23 to run. The garbage disposal 23 may include a quiet modethat runs at lower speed. Data collected or generated by the garbagedisposal 23 may be transmitted to and displayed by the control centerdisplay 16. The garbage disposal 23 may include a flow sensor to measurethe flow of particulate out of the garbage disposal 23.

The refrigerator 24 may include a refrigerator compartment that providerefrigerated storage for food and/or a freezer compartment that provideslow temperature storage for food. The refrigerator 24 may includemultiple temperature settings, child safety settings, and a statusincluding the time and temperature. The freezer compartment of therefrigerator 24 may include a temperature setting and an icemaker modefor controlling the type of ice produced by the icemaker. In addition tothese settings and statuses, the control center display 16 may providesettings and data from any of the other intelligent kitchen devices orintelligent bathroom devices. The refrigerator 24 may include a flowsensor configured to measure a flow of water to the refrigerator 24(e.g., for example for a water dispenser or icemaker), which istransmitted to the control center controller.

The water heater 25 (including tank 25 a, water supply 25 b, andoptionally fuel supply 25 c) may provide heated water to the otherappliances. The water heater 25 may include a temperature setting thatdescribes the target temperature for the water in the water heater. Thewater heater 25 may include a volume setting that defines an amount ofwater that is heated in the water heater 25. The status of the waterheater 25 may be transmitted to and displayed by the control centerdisplay 16. The water heater 25 may include a flow sensor configured tomeasure a flow of water in or out of the water heater 25, which istransmitted to the control center controller.

The water filter 26 may filter water before the water is provided to anyof the other kitchen devices. The water filter 26 may include varioussettings including filtering modes that target particular contaminants.For example, the water filer 26 may include a lead filtering mode whichmay target removing lead from the water, a bacteria filtering mode whichmay target removing bacteria from the water, or specific particulatefiltering mode for removing a specific particulate from the water. Thestatus of the water filter 26 may be transmitted to and displayed by thecontrol center display 16. The water filter 26 may include a flow sensorconfigured to measure a flow of water in or out of the water filter 26,which is transmitted to the control center controller.

Each of the intelligent kitchen devices is discussed in more detailbelow. The at least one home hub communication device may be anindependent device or integrated with any of the appliances. In oneexample a home hub communication device 29 may be integrated with acounter (e.g., retractable to move below or above the counter surface).Additional, different or fewer components may be included.

Any of the intelligent bathroom devices may interact with any of thekitchen devices. For example, data collected at one or more bathroomdevices may be analyzed to determine a command, a setting, or a displayat one or more of the kitchen devices, and vice versa. For example, theidentity of a user at a bathroom device may be sent to a kitchen devicefor accessing configuration or preferences for the user upon approachingthe kitchen device.

Communication Network

FIG. 3 illustrates a communication network for the example sets ofappliances of FIG. 1 and/or FIG. 2. The communication network mayinclude a server 13, a network device 14, and a communication bus orlocal network 15. The communication bus or local network 15 may beconnected to one or more of any combination of the intelligent mirror(mirror assembly) 1, the programmable (automated) shower 2, the bathtubsensory device 3, the bathtub level device including the drain 4 andfaucet 5, the intelligent (automated) toilet 6, the toilet (automated)seat 7, the sink faucet (automated sink) 8, light source 9 a and lightguides 9 b, a kitchen appliance 11, and a water system 12. Additional,different, or fewer components may be included.

The server 13 may be a cloud device configured to communicate withmultiple network devices 14 located in multiple locations (e.g.,different homes or businesses). The server 13 may implement a cloudservice that coordinates and analyzes data from the multiple networkdevices 14 affiliates with multiple appliances.

The network device 14 may be a standalone device (e.g., having adedicated power supply, speaker, and/or microphone) as a home hubcommunication device. Alternatively, the network device 14 may beintegrated with one or more of the appliances.

In one example, the analysis of data occurs primarily at the networkdevice 14, which may be referred to as the local analysis embodiments.In another example, the analysis of data occurs primarily at the server13 or another remote device, which may be referred to as the remoteanalysis embodiments. Hybrid embodiments may include a combination ofdata analysis at the network device 14 and the server 13.

Regarding the local analysis embodiments, the network device 14 receivesdata collected at appliance X and performs an analysis of the data togenerate a command for appliance Y. The analysis may include determiningan identity of the user of appliance X, a temporary state of the user ofappliance X, or a command from the user of appliance X. An exampleidentity of the user may include an identifier for the user (e.g.,username, user number, user code). An example temporary state of theuser may include drowsiness, complexion, sickness, or mood. An examplecommand from the user may turn on appliance Y or change a setting forappliance Y.

Regarding the remote analysis embodiments, the network device 14 maypackage or pre-process the data in a predetermined format and transmitthe data to the server 13. The network device 14 may filter the dataaccording to type. Example types include audio data, image data,position data, biometric data, ambient data, or other types. The networkdevice 14 may select a particular type of data to send to the server 13based on the types of appliances associated with the network device 14.That is, the network device 14 may sort and select data collected atappliance X, for use with appliance Y, according to the capabilities orconfiguration of appliance Y, and send the selected data to server 13.In turn, the server 13 sends the selected data to appliance Y inresponse to the capabilities or configuration of appliance Y.

For image data, the network device 14 may analyze an image of at least aportion of the user. For position data, the network device 14 maydetermine a position of the user through analysis of the image (e.g.,pattern matching or line detection) or through distance based sensorsbased on proximity. For biometric data, the network device 14 maycollect temperature data (e.g., heat signature) from a temperaturesensor or infrared sensor, fingerprint data from a fingerprint sensor,or eye data from a retina scanner. For ambient data, the network device14 may collect temperature, humidity, or other environmentalinformation.

The network device 14 may package the data in a predetermined format andtransmit the data to the server 13. The predetermined format may bespecific to the type of data (e.g., a particular file format). In oneexample, the collected data includes voice commands and thepredetermined format is an audio file. The predetermined format may bean audio encoding format (e.g., Moving Picture Experts Group (MPEG)standard, MPEG-2, mp3, wave file or other format).

In addition to being encoded in a particular audio format, the recordedaudio may include a predetermined syntax. The voice commands may includeany combination of summons commands, request commands, device functioncommands, a skill command, and other commands.

The summons command may include a trigger word or voxel to address thehome hub communication device. The trigger word may include a userspecified name for the home hub communication device or a brand name forthe home hub communication device or a generic name (e.g., hub or homehub) for the home hub communication device. In another example, thetrigger word may include a class of appliance or an associated room forthe appliance. The skill command may include device identifiers anddevice functions. The device identifier includes a code or a word thatdescribes the target device for the skill command. An examplepredetermined syntax for the voice command may be [summons] [skill] or[summons] [device identifier] [device function]. An examplepredetermined syntax for the voice command that specifies the brand ofthe appliance may be [summons] [brand] [skill] or [summons] [brand][device identifier] [device function]. An example predetermined syntaxfor the voice command that specifies the class of the appliance may be[summons] [bathroom] [skill] or [summons] [bathroom] [device identifier][device function].

The device identifier may include multiple components. The deviceidentifier may include a component identifier and a sub-componentidentifier. The component identifier may describe any of the appliancesdescribed herein. The sub-component identifier may describe a portion ofthe appliances. For example, for the component of a shower, each showersprayer is a sub-component, and for the component of a sink, the hot andcold water valves may be a sub-component. The device function commandmay be a command to apply to the component and/or the sub-component. Forexample, for the component of a shower and shower sprayer asub-component, the device function may include a level setting for theshower sprayer, and for the component of a sink, the device function maybe the temperature defining a combination of the hot and cold levels.

In one example, the summons command is omitted. For example, thepredetermined format for the home hub communication device or thenetwork device 14 may include a one-word theme control to communicatefrom one appliance to another appliance. For example, when theintelligent mirror 1 receives a word indicative of a local appliance(e.g., lighting, temperature, sound, etc.) the following words areapplied directly to that local appliance.

The appliances may also be arranged in scenes, groups, and routines. Ascene is a collection of devices that respond differently to a singlecommand. An example scene may be “good morning” that causes the fan toturn on, the shower to start at a set temperature, and the lights toturn on. Another example scene may be “lights on” that causes theceiling lights to turn on as well as the light on the mirror. A group isa collection of devices that respond in a similar fashion to one or morecommands. An example group of devices may be all the lights. Anotherexample group of devices is the door locks in the home. The group may betriggered to turn on all of the lights or lock all of the doors. Aroutine is a time sequence that of events that is applied to one or moreappliances. For example, the shower may be started and then, after afirst predetermined time period (e.g., two minutes) after the shower isstarted, the lights at the mirror are turned on, after a secondpredetermined time period (e.g., 10 minutes), the intensity, brightness,or color temperature is of the light is changed.

Another example predetermined syntax for the voice command may be[scene], which may access a scene access table to convert the scene tomultiple full commands (e.g., [summons] [skill] or [summons] [deviceidentifier] [device function]). Another example predetermined syntax forthe voice command may be [group] [device function], which may access ascene access table to convert the scene to multiple full commands (e.g.,[summons] [skill] or [summons] [device identifier] [device function]).Another example predetermined syntax for the voice command may be[routine], which may access a scene access table to convert the scene toa sequence of full commands (e.g., [summons] [skill1] or [summons][device identifier1] [device function1], [delay], [summons] [skill2] or[summons] [device identifier2] [device function2]).

The appliances may communicate using a master and slave model. Themaster device may be defined as the device that directly communicateswith the server 13 and receives voice commands from the user, and theslave device may be defined as the device that receives instructionsfrom the server 13 in response to the voice commands router to theserver 13 through the master device. In some embodiments, the networkdevice 14 is the master device and one or more of the appliances areslave devices.

In another example, the network device 14 is omitted, or otherwiseprovides only network functionality, and one of the appliances operatesas the master devices while one or more other of the appliances operatesas the slave device. The appliances may be configured to perform apriority order algorithm for determining which of the appliances aredesignated as master devices and which of the appliances are designatedas slave devices. The priority order algorithm may be based on one ormore priority techniques including order of connection, order ofinstallation, order of user preference, order of manufacturerpreferences, and/or user proximity preference.

The order of connection may indicate the first appliance that connectsto the network device 14 or server 13. The network device 14 or theserver 13 may assign master status or slave status to the appliancesupon connection to the network. For example, when an appliance connectsto the network, the appliance may send a connection request to thenetwork device 14 or server 13. The connection request may include adevice identifier for the appliance and/or a list of device functionscompatible with the appliance. In response to the connection request,the network device 14 or the server 13 may return a status such as aslave status or a master status to the appliance. For the order ofconnection technique, when the connection request is the firstconnection request or only active connection request, the network device14 or server 13 return a master status to the appliance. When theconnection request is not the first connection request or not the onlyactive connection request (i.e., another active connection exists), thenetwork device 14 or server 13 may return a slave status to theappliance. The appliance follows a master mode in response to the masterstatus or a slave mode in response to the slave mode.

In another example for the order of connection technique, the networkdevice 14 or the server 13 may assign master status or slave status tothe appliances based on internet protocol (IP) address. On the network,dynamic IP addresses may be assigned in numeric sequence according tothe order that the appliances connect to the network. The network device14 or the server 13 may receive a connection request from an applianceand the connection request may include the IP address for the appliance.The network device 14 or the server 13 may analyze the IP address andreturn a master status or a slave status to the appliance in response tothe analysis of the IP address. In one example, the network device 14 orthe server 13 may store the received IP addresses in memory and comparethe received IP addresses. The network device 14 or the server 13 mayassign the master status to the lowest IP address and the slave statusto other IP addresses. In another example, the network device 14 or theserver 13 may assign the master status or the slave status to apredetermined IP address. The assignment of IP addresses to the masteror slave status may be defined according to user input or an externaldevice.

The order of installation technique may designate the first appliance tobe installed or powered on as the master device. The network device 14or the server 13 may determine the first appliance to be installed orpowered on based on a wireless communication (e.g., Bluetoothbroadcast). The individual appliances may record a local timestamp forthe time that the appliance is installed or powered on. The networkdevice 14 or the server 13 may determine the first appliance to beinstalled or powered on a connection request from an appliance when theconnection request includes the timestamp stored by the appliance.

The order of user preference may be set by a user input or a defaultorder. The order of user preference may specify individual appliances.For example, the order of preference may specify the intelligent mirror,intelligent toilet, intelligent shower, light guides or other devices.Any order of appliances is possible. The order of user preference mayspecify types of appliances. For example, the order of types ofappliances may specify bathroom appliances then kitchen appliances, orthe order of types of appliances may specify appliances with userinterfaces (e.g., intelligent mirror), appliances with mobile devicecontrol (e.g., intelligent toilet), appliances without user interfaces(e.g., automatic leveling bathtub), and passive appliances (e.g., lightguides and versatile tiles). Within each hierarchy in the order, asecond ordering technique may be used. For example, among the userappliances with user interfaces, the second order may specify the orderthe appliances were connected to the network.

The order of manufacturer preference may be an order of devicesspecified by the manufacturer. Each appliance may store an alphanumericcode (e.g., A, 1, or another code) that indicates the order. Theappliance may send a connection request to the network device 14 or theserver 13 including the alphanumeric code for ordering. The networkdevice 14 or the server 13 may order the codes received from multipledevices. The highest code may be designated master status and othercodes may be designated slave status.

The user proximity technique may dynamically change the master and slavestatus according to the presence of the user. For example, as eachappliance detects the proximity of the user through a proximity sensoror another type of sensor, the appliance may report the proximity to thenetwork device 14 or the server 13. The network device 14 or the server13 may assign a master status to the appliance with the closestproximity to the user. Other devices may be assigned a slave status. Themaster status changes dynamically. As the user moves from one device toanother, the master status is updated. For example, when the user movesfrom near appliance X to near appliance Y, the network device 14 or theserver 13 may change the status of appliance X from master to slave andchange the status of appliance Y from slave to master.

In the master mode, one appliance can be configured to lead a group ofappliances. The master device may be the only device that receives audiocommands from the user. Audio commands at other appliances in the slavemode may be ignored. The master mode may enable a microphone of theappliance, and the slave mode may disable a microphone of the appliance.An appliance in master mode may issue instructions to other appliancesin slave mode.

In addition to exchanging information with each other, the appliancesmay exchange information with an external device 30 through the networkdevice 14 and or the server 13. The external device 30 may be associatedwith a manufacturer (e.g., the appliance manufacturer), an insuranceprovider, a utility provider or another entity.

The external device 30 may be configured to compile and analyze dataconnected by the appliances and shared through the network device 14 andthe server 13. In one example, the network device 14 generates a reportin response to the feedback data, and sends the report to the externaldevice 30. The external device 30 may provide the user with servicebenefits in exchange for the shared data. The external device 30 maycompile feedback data for multiple types of appliances and/or multiplelocations for the appliances. The external device 30 may store thefeedback data in association with the geographic locations of theappliances that collected the feedback data. The compiled data or thereport may be indicative of geographically related events such as watercontainments, illnesses, or pollution.

The manufacturer, using the external device 30, may collect data relatedto usage, maintenance, or malfunction. The usage data describes when theuser uses the appliance (e.g., a time of day or day of the week) and howthe user uses the appliance (e.g., the shower door is closed, the waterfaucet is turned on). The external device 30 may calculate from theusage data how often and the duration a particular feature of anappliance is used. For example, for the toilet, the external device 30may determine a number of seated users and a number of standing users.Features may be added or removed from a device based on the usage data.The maintenance data may describe when maintenance is applied to theappliance such as a consumable is replaced (e.g., water filter, seal,disinfectant solution) or when a maintenance provider visits theappliance. The malfunction data may include errors that are logged bythe appliance. Errors may include electronic errors with the controller,water leaks or other plumbing errors, or communication errors with thenetwork. The external device 30 may provide alerts to the user based onthe collected data. Example alerts based on maintenance data maydescribe when a consumable should be reordered. The network device 14may automatically reorder the consumable.

The network device 14 (e.g., a home hub communication device orappliance with integrated a home hub communication device) including aspeaker and the external device 30 may coordinate to provide serviceassistance to the user. After diagnostics are performed at the externaldevice 30 based on data collected at the appliances, the external device30 provides feedback to the network device 14 in the form ofannouncement of a service call and/or scheduling, do-it-yourselfinstructions for the user to perform maintenance on the appliance, orerror codes for the user to apply to the appliance or provide to atechnician. The external device 30 may also send appropriate diagnosticroutines, software upgrades, firmware upgrades or other local settingsto the network device 14.

The insurance provider, through the external device 30, may collect datarelated to user habits. Example habits include frequency of sanitaryactions such as washing hands, or frequency of risk taking actions suchas showering. The utility provider may collect data related to waterusage at the appliances at different times of day in order to pricewater costs at different levels during the day. The pricing scheme mayencourage water to be used more evenly throughout the day, whichimproves overall water conservation.

A collection of all the settings for the one or more of the appliancesmay be stored by the network device 14 or the server 13. The collectionof settings may be a user passport that is transferrable to differentlocations through the server 13 (e.g., transferred to external device30). The different locations may be different houses, hotels, or otherrooms. In other words, the collection of settings for the appliances inFIG. 1 and/or FIG. 2 may be saved and stored in association with theuser. When the user travels to another location such as a hotel room,the settings are provided on appliances at the other location. Theexternal device 30 may access the user passport based on user identitysuch as a credit card, communication with the user's phone, a detectionof the user's entity, or a code directly entered at the other location.

Appliance Coordination

FIG. 4 illustrates a flow of data between appliances for providingservices to a user, according to an exemplary embodiment. A networkdevice 14 communicates with a first appliance 31 and a second appliance32. The first appliance 31 may provide services to the user 33 based ondata collected at the second appliance 32, or vice versa. Additional,different, or fewer components may be included.

The network device 14 may include a data collection interface 36, acontroller 34, and an output interface 38. The data collection interface36 and/or the output interface 38 may be configured to communicate withthe first appliance 31 and/or the second appliance 32 using a wirelessnetwork or a personal area network. Example wireless networks includecellular networks, the family of protocols known as Wi-Fi or IEEE802.11, the family of protocols known as Bluetooth, or another protocol.The cellular technologies may be analog advanced mobile phone system(AMPS), the global system for mobile communication (GSM), thirdgeneration partnership project (3GPP), code division multiple access(CDMA), personal handy-phone system (PHS), and 4G or long term evolution(LTE) standards, or another protocol. Example wireless networks mayinclude a wireless mesh network (e.g., Bluetooth mesh). The wirelessmesh network many include may to many communication. For example, theappliances of FIG. 1 and/or FIG. 2 may be directly connected to form amesh network. Example personal area networks may include the family ofprotocols known as Bluetooth low energy or another low energy protocolconfigured to maintain a predetermined distance range under apredetermined power consumption level. The wireless mesh network mayinclude one or more nodes for appliances using Bluetooth low energy.

In one implementation, the network device 14 communicates with at leastone water consuming device. Example water consuming devices include theprogrammable shower 2, the bathtub sensory device 3, the bathtub leveldevice including the drain 4 and faucet 5, the intelligent toilet 6, theautomated sink 8, the kitchen appliance 11, and the water system 12. Thewater consuming device is connected to a water supply or plumbingsystem. The water consuming device may include at least one sensorassociated with the water supply such as a flow sensor that measures theflow of water or other types of sensors.

The data collection interface 36 is configured to receive user data fromat least one water consuming device (e.g., the first appliance 31 and/orthe second appliance 32). While only unidirectional communication isillustrated in FIG. 4, the data collection interface 36 may connect withboth the first appliance 31 and the second appliance 32 forbidirectional communication. The user data may describe sensor datacollected at the first appliance 31 and/or the second appliance 32,entry data entered by the user at the first appliance 31 and/or thesecond appliance 32, logged data recorded at the first appliance 31and/or the second appliance 32, or configuration data accessed the firstappliance 31 and/or the second appliance 32.

The sensor data, or user data, is collected at the first appliance 31and/or the second appliance 32. The sensor data may describe the spatialposition of the user. For example, the sensor data may be collected by amotion sensor or a proximity sensor. The sensor data may includethree-dimensional coordinates of the spatial position of the user of aportion of the user (e.g., feet, hands, or body). The sensor data may becollected by an eye gaze sensor that determines a line of sight orangular orientation of the face of the user. The sensor data maydescribe a gesture or movement made by the user. Example, gesturesinclude pointing handwaving, or directional movements such as raising orlowering a hand to change a setting. The sensor data may include aninfrared or laser scan of the area near the first appliance 31 and/orthe second appliance 32.

The sensor data may be collected by a camera or image collection device.The sensor data may be analyzed using an image processing technique todetermine the color, brightness, or hue of the user. Alternatively, theimage processing technique may determine the size, shape, gender, age,or other demographic data for the user. The sensor data may describe thestate of the body of the user using another characteristic such as heartrate, temperature, the presence of sweat, or odor.

The entry data can be entered by the user at the first appliance 31and/or the second appliance 32. The entry data may include login data orlogin credentials (e.g., a username and certifying information). Theentry data may include a selection for operating the first appliances31. The entry data for operating the first appliance 31 may be used tooperate the second appliance 32. For example, a temperature setting forthe automated shower 2 may be indicative of user preferences and used todetermine a temperature setting for the automated sink 8.

The logged data can be recorded at the first appliance 31 and/or thesecond appliance 32. The logged data may describe the habits of theusers over time. The logged data may include device settings associatedwith time stamps. The network device 14 may analyze the logged data toidentify trends in the historical data. For example, a temperaturesetting may tend to be warmer in winter months and lower in summermonths. The network device 14 may calculate a season factor foradjusting the temperature setting at the second appliance 32 based onhistorical data collected at the first appliance 31.

The configuration data accessed the first appliance 31 and/or the secondappliance 32 may relate to particular features of the appliance. Forexample, a particular user may use a mute setting to disable a speakerof the first appliance 31. The network device 14 may identify the mutesetting from the first appliance 31 and send a mute command to thesecond appliance 32 based on the user's preference.

The user data from the collection device may describe a condition of theuser. Example conditions of the user includes a temperature, a heartrate, a height, a weight, a drowsiness, a complexion, a malnutrition, ahydration, or other conditions.

The user data may be modified by the data collection interface 36 or thecontroller 34. The user data may be filtered to remove noise. Ahysteresis control algorithm may be applied to the user data. A firstthreshold may be applied to increasing data and a second threshold maybe applied to decreasing data to prevent rapid threshold crossings frombeing present in the user data. The user data may be averaged over apredetermined time period to reduce the effects of outlier data points.

The controller 34 is configured to perform an analysis of the user datafrom the at least one collection device. The analysis may compare theuser data to a threshold or profile that is indicative of a devicefunction. When the data is greater than the threshold level or matches apredetermined portion of the profile, the controller 34 generatesfeedback data for another device.

The controller 34 may average user settings or preferences for one ormore features of the at least one collection device and save the averageas the feedback data. The averaged preference for one or more featuresof the at least one collection device may include a temperature setting,a volume of water, a light intensity, or a light direction. For example,when the collection device is a shower, the controller 34 may averagethe temperature settings by all users or multiple instances of a singleuser to calculate a temperature for the feedback data.

The output interface 38 is configured to provide feedback data based onthe analysis of the user data to the first appliance 31. The feedbackdata may be applied by the water consuming appliance to set a watertemperature, a water volume, or a water pattern in response to thefeedback data. The feedback data may activate or deactivate the firstappliance 31 or a particular function of the first appliance 31.

FIG. 5 illustrates a flow chart for a method for coordination ofhousehold devices, according to an exemplary embodiment. The method maybe performed by the network device 14, or specific aspects of the methodmay be performed by the data collection interface 36, the controller 34,and the output interface 38, respectively. Additional, different, orfewer acts may be included.

At act S101, the data collection interface 36 receives sensor data fromat least one first device, the sensor data describing a state of a user.The at least one first device may be a water consuming device (e.g.,bathtub, sink, shower, toilet). The state of the user may be thepresence of the user, the position of the user, or a measuredcharacteristic of the user.

At act S103, the controller 34 performs an analysis of the sensor datafrom the at least one first device. At act S105, the controller 34generates feedback data based on the analysis of the sensor data. Thefeedback data may include an instruction to operate a second device or aparticular device function. When the state of the user is the presenceof the user, the analysis may include comparing the presence or absenceof the user to a lookup table that lists device functions that areactivated when the user is present. Example device functions that areactivated when the user is present may include a light, a microphone, ora heater. When the state of the user is the presence of the user, theanalysis may include comparing the position to a lookup table that listsdevice functions that are activated when the user is at certainpositions. Example device functions that are activated based on positionmay include turning on the water when the user is at the sink, flushinga toilet when the user is positioned away from the toilet, or turning ona light based on the position of the user. When the state of the user isa measured characteristic (e.g., temperature, color, or mood), theanalysis may include comparing the measured characteristic to a lookuptable for the measured characteristic.

At act S107, a second device is operated using the feedback data basedon the user data from the at least one first device. The second devicemay be operated in response to instructions issued by the controller 34.The second device may be a water consuming device (e.g., bathtub, sink,shower, toilet).

FIG. 6 illustrates a flow of data between appliances for controlling afirst device based on data collected by a second device, according to anexemplary embodiment. The network device 41 described with respect toFIG. 4 may be included or omitted from the example of FIG. 6. The flowof data between appliances may include multiple types of data includingbut not limited to an audio recording 43, command data 45, and a setting47.

The recording 43 may include audio data including a voice command.Example voice commands may include any combination of summons commands,request commands, device function commands, a skill command, and othercommands. The voice command may include a predetermined syntax (e.g.,[summons], [device function command]). The voice command may includenatural language. The recording 43 may be interpreted at the firstappliance 31, server 13, or at the network device 14. Alternatively, thevoice command may be interpreted at the second appliance 32 and encodedas command data 45. The command data 45 may include a data packet with adevice identifier and a function identifier. In addition, the user maydirectly enter an instruction to the second appliance 32 that is to beapplied to the first appliance 31. The direct command may be transferredbetween appliances as setting 47.

FIG. 7 illustrates bidirectional flow of data between appliances,according to an exemplary embodiment. The network device 14 describedwith respect to FIG. 4 may be included or omitted from the example ofFIG. 6. The flow of data between appliances may include multiple typesof data including but not limited to sensor data 37 and command data 45.The first appliance 31 collects sensor data 37 and sends the sensor data37 directly to the second appliance 32, or indirectly through thenetwork device 14 and/or the server 13. The second appliance 32 analyzesthe sensor data 37 collected at the first appliance 31 and sends commanddata 45 directly to the first appliance 31, or indirectly through thenetwork device 14 and/or the server 13.

User Identification

FIG. 8 illustrates another flow of data between appliances for useridentification, according to an exemplary embodiment. The network device14 communicates with the first appliance 31 and the second appliance 32.The first appliance 31 may provide services to the user 33 based on datacollected at the second appliance 32, or vice versa. Additional,different, or fewer components may be included.

The first appliance 31 is configured to collect sensor data associatedwith an identity of a user. The sensor data associated with the identityof the user may include identity data 19, biometric data 35, or sensordata 37. The identity data 39 may include a username or other identifierfor the identity of the user of the first appliance 31. The identitydata 39 may be derived from image data or proximity data that describesthe user. The biometric data 35 may include a body profile, afingerprint, facial recognition, or other data. The sensor data 37 mayinclude temperature, stress, blood pressure, wakefulness, or othertransient characteristics.

The second appliance 32 may access a configuration for the user based onthe user identity determined from the sensor data collected by the firstappliance. The device function is then defined according to theconfiguration for the user. The configuration for the user includes aheight for the device function, a temperature for the device function, atime delay for the device function, a volume for the device function, ora sequence for the device function, or other settings.

The server 13 or the network device 14 analyzes the sensor data todetermine the identity of the user. The second appliance 32 isconfigured to provide a device function to the user based on identitydetermined from the sensor data collected by the first appliance.Additional appliances may be connected as well. For example, a thirdappliance may be configured to provide a device function to the userbased on identity determined from the sensor data collected by the firstappliance. In any of the bathroom appliances described with respect toFIG. 1, the kitchen appliances described with respect to FIG. 2, orother appliances may be the first appliance 31 or the second appliance32. In another implementation, the first appliance 31 or the secondappliance 32 is a bathroom appliance connected to plumbing or the watersystem. In another implementation, the first appliance 31 or the secondappliance 32 is a water consuming appliance.

In addition to further examples of device functions described herein,the following are examples of device functions applied by the secondappliance 32 based on identification data collected at the firstappliance 31. In one example, the device function may turn on (activate)or turn off (deactivate) a power status, a standby status, or a data orwater flow to the intelligent mirror 1, the programmable shower 2, thebathtub sensory device 3, the bathtub level device for faucet 5, theintelligent toilet 6, the toilet seat 7, the sink faucet 8, the lightsource 9 a, the kitchen faucet 21, the dishwasher 22, the garbagedisposal 23, the refrigerator 24, the water heater 25, and/or the waterfilter 26.

Specific examples of device functions for the programmable shower 2 mayinclude determining a shower sequence in response to the identity of theuser determined by the server 13 based on the sensor data collected bythe first appliance 31. The shower sequence may include at least onetiming component for the user, at least one temperature component forthe user, at least one shower head selection for the user, or at leastone spray selection for the user. The shower sequence may vary based onseason, weather, time of day, or day of week. When the sensor dataincludes a state of the user based on health, heart rate, temperature,or stress, the shower sequence is based on the state of the user.

Specific examples of device functions for the bathtub device may beapplied to the bathtub sensory device 3 or the bathtub level device forfaucet 5. The device function may be a flow of water, a temperature ofwater, or activating or deactivating the faucet 5. The device functionmay specify a level of the bath. The drain and faucet 5 may becoordinated (e.g., commands to release some water and add some water tothe bath) to maintain a specific level. In addition or in thealternative, the drain and faucet 5 may be coordinated to maintain aspecific temperature for the bath. The device function may cause thebathtub sensory device 3 to emit a sequence of audio or vibration to thebath. The sequence may be music or a series of vibration pulses.

Specific examples of device functions for the intelligent toilet 6 mayinclude flushing the toilet, cleaning the toilet, illuminating thetoilet, or warming a portion of the toilet. Specific examples of devicefunctions for the faucets may include a volume of water, a temperatureof water, or a time duration of water. The second appliance 32 includesthe light source 9 a configured to illuminate a specific area as thedevice function.

Specific examples of device functions for the water filter 26 configuredto filter a flow of water. The filter may be selected based on theidentity of the users. A user configuration may include a preference forwater filtering that specifies a particular type of filtering or asubstance to be removed from the water. The water filtering may also beimplemented by any faucet, the refrigerator 24, or the water heater 25.In addition, a water additive system may be implemented by any faucet,the refrigerator 24, the water heater 25, the water filter 26, oranother water consuming device. The water additive system is configuredto provide an additive to a flow of water. A user configuration mayinclude a preference for water additive that specifies a particularadditive for the water. Example additives described herein includefluoride, vitamins, or flavoring.

FIG. 9 illustrates a control system 301 for the appliances or home hubcommunication device, according to an exemplary embodiment. The controlsystem 301 can be implemented by any of the appliances in FIG. 1 and/orFIG. 2, the network device 14 or the server 13. The control system 301may include a processor 300, a memory 352, and a communication interface353 for interfacing with devices (e.g., appliances, network device 14,or server 13, respectively) or to the internet and/or other networks346. The components of the control system 301 may communicate using bus348. The control system 301 may be connected to a workstation or anotherexternal device (e.g., control panel) and/or a database for receivinguser inputs, system characteristics, and any of the values describedherein. Optionally, the control system 301 may include an input device355 and/or a sensing circuit in communication with any of the sensors.The sensing circuit receives sensor measurements from as describedabove. Optionally, the control system 301 may include a drive unit 340for receiving and reading non-transitory computer media 341 havinginstructions 342. Additional, different, or fewer components may beincluded. The processor 300 is configured to perform instructions 342stored in memory 352 for executing the algorithms described herein.

FIG. 10 illustrates a flow chart for the operation of one appliancebased on identification data collected at another appliance, accordingto an exemplary embodiment. The acts of the flow chart may be performedby any combination of the home hub communication device, the networkdevice 14 or the server 13. Portions of one or more acts may beperformed by the appliance. Additional, different of fewer acts may beincluded.

At act S201, control system 301 (e.g., through communication interface353) receives sensor data collected by the first appliance 31. Thesensor data is associated with an identity of a user. The sensor datamay be received through wireless communication and encoded with atimestamp, a device identifier for the first appliance 31, and a datatype identifier for the type of sensor that collected the sensor data.The sensor data may be collected by a combination of sensors. Thecommunication interface 353 may include circuitry, a module, or anapplication specific controller as a means for receiving sensor datacollected by the first appliance.

At act S203, the control system 301 (e.g., through processor 300)analyzes the sensor data from the first appliance to determine theidentity of the user. The control system 301 may match a physicalcharacteristic described in the sensor data with a stored value for theuser. The control system 301 may match a series of values over time, ora series of values across multiple sensors, with a stored profile forthe user. The processor 300 may include circuitry, a module, or anapplication specific controller as a means for analyzing the sensor datafrom the first appliance to determine the identity of the user.

At act S205, the control system 301 (e.g., through processor 300)accesses a user database using the identity of the user for a userconfiguration. The user configuration may include preferences for theuser applicable across multiple appliances such as whether or not toenable or disable types of device functions. The user configuration mayinclude specific settings saved for the user. The user configuration mayinclude a sequence of preferred function or a time for when particulardevice function is preferred by the user. The processor 300 may includecircuitry, a module, or an application specific controller as a meansfor accessing a user database using the identity of the user for a userconfiguration

At act S207, the control system 301 (e.g., through processor 300)generates settings for controlling generating a command for a devicefunction for a second appliance based on user configuration for theidentity of the user. The processor 300 may include circuitry, a module,or an application specific controller as a means for generating acommand for a device function for a second appliance based on userconfiguration for the identity of the user.

Intelligent Mirror

FIG. 11A (and the exploded view FIG. 11B) illustrate an example mirror101, and cabinet behind the mirror 101, including a home hubcommunication device, at least one light strip 103, and user interface105. Alternatively, the mirror 101 may communicate with another home hubcommunication device, which may be a standalone device or integratedwith another appliance. Additional, different, or fewer components maybe included.

The mirror 101 may include a mirror substrate configured to provide areflection of objects including a user. The mirror substrate reflectssubstantially all of the light that meets the mirror substrate at thesame angle the light meets the mirror substrate and/or substantiallynone of the light is absorbed or scattered. Substantially all of thelight may mean 90%, 99% or another proportion of the light.Substantially none of the light may mean 10% 1% or another proportion ofthe light. The mirror substrate may be made from polished material orfrom transparent glass that is coated with a thin layer of reflectivemetal (e.g., silver or aluminum).

The mirror 101 may include a sensor 102 configured to collect sensordata from the user. The sensor 102 may be an image collection devicewith a lens such as a digital aperture collection device (e.g., camera)or an image collection device with a charge coupled device (CCD) such asan integrated circuit formed on a silicon surface forming lightsensitive elements. The image collection device may collect images forfacial recognition of the user. The image collection device may collectimages for recognizing an image signature of the user such as the coloror shape (e.g., bone density, outline, height, and/or weight) of theuser. Other body properties may be determined from the image of the userincluding skin qualities at a cellular level, signs of hormoneimbalance, aging, sun damage, pigmentation, color, inflammation,environmental impacts, or other abnormalities. For example, the imagesof the user's teeth may be analyzed for feedback for brushing teeth. Themirror 101 detects whether brushing or flossing was sufficient andinstructs the user to continue brushing or flossing. In another example,the images of the user's muscles are analyzed to determine muscleconditions (e.g., strains, pulls, or tears) which are highlighted on thedisplay and transmitted to an exercise device for a customized workoutor to a trainer or website for customized exercise plan.

The sensor 102 may be a relative distance collection device such as aproximity sensor or a laser scanner. The laser scanner may emit one ormore laser pulses that reflect off of objects and are received by thelaser scanner. The time of flight for the laser pulses indicates thedistance to the objects. The proximity sensor may detect a presence ofan object at a predetermined distance or within a predetermined distancerange. The proximity sensor may include a microwave or radar sensor.Example predetermined distances may be 28 inches, 1 meter or anotherdistance. The range of the proximity sensor may be cone shaped.

The sensor 102 may be a temperature mapping device such as an infraredcamera for detecting a heat signature of the user. The sensor 102 may bea retina scanner configured to scan the eyes of the user. The retinascan may indicate an eye signature for identification of the user. Theretina scan may reveal the blood sugar level of the user.

The sensor 102 may be an audio sensor such as a microphone. The sensor102 may detect odors. The sensor 102 may detect volatile organiccompounds (VOCs) or other carbon based (organic) chemicals (compounds)that are indicative of odor. The sensor 102 may be an environment sensorsuch as a temperature sensor, a light sensor, or a humidity sensor. Thesensor 102 may be a remote sensor in another location (e.g., a differentroom than the mirror assembly 101.

The mirror assembly 101 comprises user interface 105, which isconfigured to receive an instruction from the user and/or display datato the user. The instruction from the user may trigger an auxiliarycommand for the auxiliary device. The data displayed at the userinterface 105 includes status data for the auxiliary device, settingsdata for the auxiliary device, configuration data for the user, or typedata for the auxiliary device.

The user interface 105 may include a touchscreen. The user interface 105may include a temporal component 107, a user input 109, and a weathercomponent 111. FIG. 12 illustrates another example user interface for amirror and cabinet, including a similar temporal component 107 andweather component 111. FIG. 13 illustrates another embodiment of theuser interface and mirror substrate integrated as a transitional userinterface and mirror. By varying an electric current through differentportions of the mirror substrate, the mirror substrate may changebetween a mirror and the user interface. A network of traces or wirescarry the varying electric current. In one example, when data isreceived from another device, the current is varied so that a portion ofthe mirror becomes a user interface. In one example, when a summonscommand is received from the user, the current is varied so that aportion of the mirror becomes a user interface.

FIGS. 14A and 14B illustrate an apparatus for a mirror including asupport housing 115, a sensor module 121, a control module housing 122,and mirror interface 120, according to an exemplary embodiment. Acontrol module 123 may be shaped to mate and/or fit inside the controlmodule housing 122. The control module 123 may include a controlinterface 130 on the underside. FIG. 14A illustrates a wider supporthousing 115 and FIG. 14B illustrates a narrow support housing 115. FIG.14B also illustrates support feet 124 for holding the control module 123to the support housing 115. Additional, different, or fewer componentsmay be included.

The sensor module 121 may include one or more of the types of sensorsthat collect sensor data in the examples described herein. The sensordata for a user is received at the mirror assembly 101, or specificallycontrol module 123, from the sensor module 121 or an external sensor.The control module 123 may include a controller configured to analyzethe sensor data and select an auxiliary command for an auxiliary devicecoordinated with the mirror assembly, the auxiliary command selected forthe user.

The auxiliary device is one of the appliances described herein. Theauxiliary device may be an adjacent device within the same room as themirror assembly 101 or less than a predetermined distance to the mirrorassembly 101. The auxiliary device may be a remote device in a differentroom from the mirror assembly 101 or greater than a predetermineddistance from the mirror assembly 101. With respect to the mirrorassembly 101, the adjacent devices may include bathroom devices such asthe intelligent mirror 1, the programmable shower 2, the bathtub sensorydevice 3, the bathtub level device, the intelligent toilet 6, the toiletseat 7, the sink faucet 8, light guides, or a fan. With respect to themirror assembly 101, the remote devices may include kitchen devices suchas the kitchen faucet 21, the dishwasher 22, the garbage disposal 23,the refrigerator 24, the water heater 25, and the water filter 26.

The control module 123 may select, based on the analysis of the sensordata, an auxiliary command for an auxiliary device coordinated with themirror assembly 101. The auxiliary command may be selected for aparticular user. The auxiliary command for the auxiliary device is basedon the instruction from the user received at the user interface and thedata displayed at the user interface includes status data for theauxiliary device, settings data for the auxiliary device, configurationdata for the user, or type data for the auxiliary device.

The analysis of the sensor data may determine an instruction received atthe user interface 105. For example, the user may enter a command forthe programmable shower 2 (e.g., select shower sequence or turn on/offshower), a command for the bathtub sensory device 3 (e.g., select a bathexperience), a command for the bathtub level device (e.g., select a bathlevel), a command for the intelligent toilet 6 (e.g., flush or close thelid), a command for the toilet seat 7 (e.g., raise or lower the toiletseat), a command for the sink faucet 8 (e.g., select a temperature orvolume), a command for light guides (e.g., a position or angle for thelight), a command for the fan (e.g., a fan speed or on/off), a commandfor the kitchen faucet 21 (e.g., select a temperature or volume), acommand for the dishwasher 22 (e.g., a cleaning mode or on/off), acommand for the garbage disposal 23 (e.g., a speed or on/off), a commandfor the refrigerator 24 (e.g., select a temperature), a command for thewater heater 25, (e.g., select a temperature), or a command for thewater filter 26 (e.g., activate or deactivate).

Settings data for the auxiliary device may be based on informationreceived at the control module 123 from another of the auxiliarydevices. For example, data collected regarding height from a showersetting may be applied to a toilet seat position, a position of the userinterface 105 on the mirror substrate or a position of a light guide.Configuration data for the user may be based on the identification ofthe user or detection of the user and preferences previously establishedfor the user. Type data for the auxiliary device may be used to apply asetting across a class of devices. For example, the user preferences mayspecify a temperature that is applied to all auxiliary devices with atemperature setting.

In one example, the control module 123 includes the componentsillustrated by FIG. 9. The control module 123 may include acommunication interface configured to send the auxiliary command to theauxiliary device.

The control module 123 may include a speaker configured to provide audiofor status data for the auxiliary device, settings data for theauxiliary device, configuration data for the user, or type data for theauxiliary device. The speaker may be movable. The control module 123 mayengage a positioning mechanism (e.g., stepper motor or solenoid) to movethe speaker toward a user. The degree of movement may depend on the taskperformed by the user, the identity of the user, the height of the user,the preferences of the user. The volume of the speaker may beconfigurable. The control module 123 may set a volume of the speakerbased on the task performed by the user, the identity of the user, theage of the user, the height of the user, the preferences of the user.For example, volume may be set proportional to age. Certain tasks mayinvoke higher volumes. For example, when the shower is running a highervolume is used.

The control module 123 may include a microphone configured to collectaudio (e.g., voice commands) for settings data for the auxiliary device,configuration data for the user, or type data for the auxiliary device.The microphone may be movable. The control module 123 may engage apositioning mechanism (e.g., stepper motor or solenoid) to move themicrophone toward a user. The degree of movement may depend on the taskperformed by the user, the identity of the user, the height of the user,the preferences of the user. The volume of the microphone may beconfigurable.

The voice commands received at the control module 123 may include deviceidentifiers that describe the auxiliary device and device functions tobe performed by the auxiliary device, which are described herein withrespect to various appliances. In addition, the voice commands mayinclude a device identifier for the mirror assembly 101, or omitting thedevice identifier may default to the mirror assembly, and a devicefunction for the mirror assembly 101. Example functions performed at themirror assembly 101 may include control of the lights, control of thelight guide, selection of the collected data and selection of thedisplayed data.

The control of the lights (e.g., light strip 103) may include the colorof the lights, brightness of the lights, intensity of the lights, orschedule for the lights. The control of the light guide may include anangle or position for the light is determined based on the auxiliarycommand selected for the user. For example, the voice command mayinstruct the light guide to illuminate handwashing in response to thevoice command. The selection of the collected data may enable or disableone or more sensors. The selection of the displayed data may enable ordisable the display of external data (e.g., weather) or data receivedfrom auxiliary devices.

In one example, the control module 123 may include an inductive chargingassembly including one or more inductive chargers. The inductivechargers may include coils of wire configured to produce magnetic fieldsthat charge batteries in mobile devices. The mobile device may be placedon control module 123 to charge the battery of the mobile device. Acharging assembly may be incorporated into any of the appliancesdescribed herein.

FIG. 15 illustrates a control interface 130 for the communication moduleof FIGS. 14A and 14B, according to an exemplary embodiment. The controlinterface 130 may include one or more tactile switches such as a volumecontrol 137, a capacitive light control 133, and a wireless networktoggle button 135. The capacitive light control 133, or any of theinputs to the control interface 130 may include a capacitive sensorresponsive to touch. Adjacent to the control interface 130 may be one ormore speakers 132, and an array of light emitting diodes (LEDs)including nightlight LEDs 134 and home hub communication indicators 136.

The control interface 130 may provide a command for the auxiliarydevice. As shown, volume control 137 and/or capacitive light control 133may be applied to auxiliary devices. Otherwise, the control interface130 may include an activation for turning an auxiliary device on andoff. The wireless network toggle button 135 may also turn wirelesscommunication with an auxiliary device on and off.

FIG. 16 illustrates a cabinet including a panel 140 for the controlmodule 123 of FIGS. 14A and 14B, according to an exemplary embodiment.The cabinet, including one or more storage spaces or compartments, maybe coupled to the mirror interface 120. Alternatively, the mirrorinterface 120 may be omitted. The panel 140 may include a universalserial bus or similar communication port 141, a microphone activated ormuted LED 143, an inquiry LED 145, and a proximity sensor 147.

The communication port 141 may connect to a laptop or smart phone forsetup or configuration of the control module 123. In one example,configuration requires a hard wired connection. The communication port141 may be used as a charging port for a phone, a shaver, a hair dryer,or other chargeable appliance. The communication port 141 maycommunication with supplemental or replacement lights, speakers ormicrophones.

The microphone activated or muted LED 143 is an indicator that indicateswhen the microphone is in use. The user may activate or deactivate themicrophone using a voice command or other user input. The inquiry LED145 may indicate that voice commands are being received. For example,the inquiry LED 145 may illuminate in response to the summons command.Thus, the inquiry LED 145 indicates that the home hub communicationdevice has correctly identified that voice commands are being received.The proximity sensor 147 may be used in coordination with sensor 102.

Intelligent Toilet

FIG. 17 illustrates a 150 including a home hub communication device anduser interface, according to an exemplary embodiment. The toilet 150 isa plumbing fixture including a basin 156, a lid 158, a seat 154, and ahousing. The seat 154 may be controlled by an electromechanical liftingand closing device configured to automatically open and close the seat.The toilet 150 may receive a fluid input (e.g., water) from a fluidsupply line and may deliver the fluid to basin 156 via an internal fluidconnection (not shown). The toilet 150 may receive electrical energyfrom an external or internal energy source. For example, the toilet 150may connect to a standard residential power outlet (e.g., 120 V 60 Hz),a power generator, or other power source (e.g., batteries, capacitors,etc.). In some embodiments, the toilet 150 functions as a toilet, usingthe fluid to flush, rinse, or otherwise clean basin 156. The toilet 150may provide a variety of flushing options configured to carry out theflushing process. For example, one flushing option may clear basin 156,automatically wash basin 156, and then refill basin 156 for subsequentuse. Other flushing options may automatically clean basin 156, sanitizebasin 156, or initiate a process to reduce or eliminate odor. The toilet150 may provide one or more flushing options configured to use variousamounts of water or power during the flushing cycle. In someembodiments, the toilet 150 may include a bidet wand, a cleaningelement, a fan, or other optional features. In some embodiments, thetoilet 150 functions as a bidet, delivering the fluid to the bidet wand.When the toilet 150 functions as a bidet, the seat may be automaticallylifted using the electromechanical lifting and closing device. Thetoilet 150 may provide a variety of bidet control options includinguser-customizable spray patterns and an adjustable spray pressure,temperature, or position. The toilet 150 may automatically clean andsanitize the bidet wand using an internal ultraviolet sanitizing light.In some embodiments the toilet 150 functions as a combination toilet andbidet, providing both functionalities.

The toilet 150 is shown to include a housing. The housing may enclose(e.g., surround, encapsulate, contain, etc.) some or all of the othercomponents of the toilet 150 (e.g., plumbing components, electricalcomponents, mechanical components, etc.). The toilet 150 may providesupport for other components, thereby allowing such components to bepositioned for proper operation of the toilet 150 as described herein.In some embodiments, housing may protect the internal components fromexternal sources of damage (e.g., physical damage, chemical damage,electrical damage, etc.). In some embodiments, housing may be a singleshell encapsulating all of the toilet 150. In other embodiments, thehousing may include multiple shells. For example, FIG. 17 shows thehousing divided into multiple sections. A frontal housing section 152 ais shown supporting basin 156. A rear housing section 152 b is shownsurrounding another portion of fixture the toilet 150. The rear housingsection 152 b may include a control module 157 (e.g., the control system301 of FIG. 9), a display 153, and a communication module 155.

FIG. 17 illustrates an all in one or integrated embodiment of theintelligent toilet 150. FIG. 18A illustrates a distributed embodiment ofan intelligent toilet 180 including modular components for an automatedcovering assembly 170, a touchless flush device 177, and an automaticcleaning system 175.

The automatic cleaning system 175 may include a valve that releases asolution in the tank of the intelligent toilet 180. The solution flowsto the basin 156 to clean the basin at the point illustrated in FIG.18A. The automatic cleaning system 175 may include a sprayer that spraya solution into the basin 156. The solution may be cleaning solution ordisinfected solution.

The touchless flush device 177 may be an electronic flush mechanism forsending a flush command to the intelligent toilet 180. The touchlessflush device 177 may include a remote detector (e.g., a disk or puck)and a mechanical flusher coupled to the basin 156. The touchless flushdevice 177 may include a gesture sensor, which is a type of proximitysensor configured to detect a gesture by the user. The gesture may bewaving a hand or otherwise placing a hand near the touchless flushdevice 177.

FIG. 18B illustrates a control system and communication system for theintelligent toilet 180, according to an exemplary embodiment. Thecontrol system includes a controller or control module 183 (e.g., thecontrol system 301 of FIG. 9), a dedicated remote 185, a remoteapplication on phone 181, a heater 182, a light strip 186, a sensor 188,and a mechanical flush lever 187. FIGS. 19A and 19B illustrate anautomated toilet covering assembly 170 including lid 171, a seat 172,which may be integrated with toilet in a single construction, a hingeassembly 173, and one or more electrical connectors 174. Additional,different or fewer components may be included. Herein the control module183 refers to both the controller of the all in one or integratedembodiment of the intelligent toilet 150 and the distributed embodimentof an intelligent toilet 180 including modular components.

The control module 183 coordinates any combination of the automatedcovering assembly 170, the touchless flush device 177, and the automaticcleaning system 175. The control module 183 may send commands toautomated covering assembly 170.

The control module 183 may send commands to the touchless flush device177 to operate the mechanical flush lever 187. The control module 183may send commands to the automatic cleaning system 175 to initiate acleaning process (e.g., spray or inject cleaning solution). Thededicated remote 185, or the remote application on phone 181, mayreceive user input for any of the commands for the control module 183.

The intelligent toilet 180 may include a sensor, such as the sensor 188,configured to collect sensor data from the user. The sensor 188 may bean image collection device with a lens such as a camera or an imagecollection device with a charge coupled device (CCD) such as anintegrated circuit formed on a silicon surface forming light sensitiveelements. The image collection device may collect images of the user.The image collection device may collect images for recognizing an imagesignature of the user such as the color or shape (e.g., bone density,outline, height, and/or weight) of the user. The image collection devicemay be a gesture sensor.

The sensor 188 may be a relative distance collection device such as aproximity sensor (e.g., sensor 188) or a laser scanner. The laserscanner may emit one or more laser pulses that reflect off of objectsand are received by the laser scanner. The time of flight for the laserpulses indicates the distance to the objects. The proximity sensor maydetect a presence of an object at a predetermined distance or within apredetermined distance range. The proximity sensor may recognizemultiple types of gestures. For example, one gesture may correspond toan ecological flush and another gesture may correspond to a normalflush.

The proximity sensor may emit and/or detect a beam of light (e.g.,infrared light). Breaking the beam of light or interrupting the beam oflight signals that the user is present. Breaking the beam of lightincludes placing a body part or object in the path of the beam of lightso that the proximity sensor does not receive the beam of light. Thebeam of light may be near the floor or base of the intelligent toiletsuch that the beam of light can be easily broken by the user's foot.

The sensor 188 may be a temperature mapping device such as an infraredcamera for detecting a heat signature of the user. The sensor 188 may bea retina scanner configured to scan the eyes of the user. The sensor 188may be a fingerprint sensor. The sensor 188 may be an audio sensor suchas a microphone. The sensor 188 may detect odors or VOCs. The sensor 188may be an environment sensor such as a temperature sensor, a lightsensor, or a humidity sensor. The sensor 188 may be a remote sensor inanother location (e.g., a different from than the intelligent toilet).The sensor 188 may be a leak sensor to detect a water leak in theintelligent toilet. For example, the sensor 188 may detect a water level(e.g., tank, basin) that is analyzed by the control module 183 todetermine when a possible leak is occurring.

The sensor 188 may collect data on human waste. The sensor data may beused for a urinalysis or urine screen. The sensor data may include a pHlevel, an indication of the presence or absence of one or more ketones,an indication of the presence or absence hormones indicative ofpregnancy, an indication of the presence or absence of blood cells orbacteria indicative of a urinary tract infection, an indication of thepresence or absence of drugs, an indication of the presence or absenceof a particular type of cells (e.g., cancer cells) or other substances.The sensor 188 may include a light sensor (e.g., infrared light orlaser) to test blood sugar levels or oxygen levels, which may be used tomonitor diabetes. The sensor 188 (e.g., a camera) may collect data forstool samples. The control module 183, the home hub communicationdevice, or the external device 30 may perform an image analysis on thestool samples.

The sensor 188 may be configured to output data that describes when theintelligent toilet 180 is in use. The sensor data may be binaryincluding one value to indicate the intelligent toilet is in use andanother value to indicate the toilet 180 is not in use.

The sensor 188 may be incorporated in the automated covering assembly170. A weight sensor (e.g., pressure sensor) may detect the user'sweight, or a rough estimate of the user's weight, when seated on thetoilet seat. The control module 183 may calculate the weight of the userfrom the sensor data. The control module 183 may determine a demographictype of user such as man versus woman or adult versus child. The controlmodule 183 may identify the user. For example, in a household, weightsof the people are distinct enough to recognize the identity of theindividual users. The control module 183 may learn the identities bymeasuring the weight of a known user. For example, the user may connectusing the dedicated remote 185 or the phone 181, and the control module183 records a weight reading from the sensor. The weight is stored bythe control module 183 and when a subsequent user is seated, asubsequent reading is compared to the stored reading to determinewhether the identity of the user matches. Any number of readings may betaken for any number of users.

The sensor 188 may be a capacitance sensor or an array of capacitancesensors that detect the legs and thighs of the user. The sensor data mayindicate whether or not the user is seated. The sensor data may indicatewhether a leg is lifted and for how long. The sensor data indicatesusage patterns of the intelligent system.

The automated covering assembly 170, the seat 172, or alternativelyintelligent toilet 150, may include sensors such as biometric sensors todetect biological characteristics or biometrics of the user. The sensorsmay include the weight sensor configured to collect weight data for theuser. The sensors may include a sensor for determining height, and thecontrol module 183 may calculated BMI from height and weight.

The biometric sensors may include body composition sensors. A bodycomposition sensor may measure muscle mass and/or body fat percentageusing a bioelectrical impedance analysis. Alternatively, the bodycomposition sensor may measure body water percentage contained in thecells and tissues using the bioelectrical impedance analysis, or bonemass using the bioelectrical impedance analysis. Changes in muscle orfat in certain parts of the body change the impedance of those parts ofthe body. The impedance may be detected by the control module 183sending a low level electrical current through the user's body andmeasuring changes in the low level electrical current when it returns toa sensor or the control module 183. The current may enter and returnfrom the user's body through the seat 172 (e.g., legs) or the coverassembly 170 (e.g., back), or the current may enter through one of theseats 172 or the cover assembly 170 and return through the other of theseat 172 and the cover assembly 170.

The biometric sensors may include a metabolic sensor that measures thebasal metabolic rate (BMR) or minimal rate of energy per time expendedby the user at rest while seated at the toiled. The BMR describes thecalories needed by the body to rest.

The biometric sensors may include a photoplethysmogram (PPG) configuredto optically obtain a volumetric measurement of an organ. The PPG may beconfigured to measure the heart rate of the user (e.g., through theskin), cardiovascular pulse wave velocity, respiration, or anothercharacteristic of the user.

In one or more of these examples, the sensors detect biologicalcharacteristics such as body composition, heart rate, temperature, orblood pressure of the user. The biometric sensors, taken alone or incombination with the weight sensor, may provide a user signature foridentifying the user.

Other sensors may be used to determine the identity of the user. Thesensors may include an IR sensor, which is included in the automatedcovering assembly 170, may generate a seat signature for the user asseated. Similarly, an IR sensor in the front of the basin may generate aheat signature of the user's feet. A proximity sensor may detect thepresence or absence of feet and/or gestures made by the feet.

The sensor 188 may be incorporated in the touchless flush device 177both for detecting the gesture command for initiating a flush anddetecting the mechanical flush lever 187 (e.g., for acknowledging theflush has occurred, detecting the number of flushes, or activating acleaning sequence). In addition or in the alternative, the sensor 188may be incorporated in the automatic cleaning system 175. The sensor 188may detect the cleaning sequence or detect commands for initiating thecleaning sequence.

The control module 183 may send commands based on the sensor data tovarious components of the intelligent toilet 180. When the user isdetected by the proximity sensor, the intelligent toilet 180 is placedin a standby mode. The standby mode may include any combination ofopening the lid, raising the seat, warming the foot warmer, turning onthe display, or turning on the remote 185 or activation thecorresponding application on the phone 181.

In addition to commands for the intelligent toilet 180, the controlmodule 183 may select, based on the analysis of the sensor data, anauxiliary command for an auxiliary device coordinated with theintelligent toilet 180. The auxiliary command may be selected for aparticular user. The auxiliary command for the auxiliary device is basedon the instruction from the user received at the user interface and thedata displayed at the user interface includes status data for theauxiliary device, settings data for the auxiliary device, configurationdata for the user, or type data for the auxiliary device. For example.the seat may be raised as a function of the identity of the user and/ortime of day (e.g., gender, pattern of use).

The analysis of the sensor data may determine an instruction received atthe user interface of the intelligent toilet 180, through gestures orother commands received by the sensors (e.g., voice commands). Forexample, the user may provide instructions including device identifiersand device functions for one or more of a command for the programmableshower 2 (e.g., select shower sequence or turn on/off shower), a commandfor the bathtub sensory device 3 (e.g., select a bath experience), acommand for the bathtub level device (e.g., select a bath level), acommand for the sink faucet 8 (e.g., select a temperature or volume), acommand for light guides (e.g., a position or angle for the light), acommand for the fan (e.g., a fan speed or on/off), a command for thekitchen faucet 21 (e.g., select a temperature or volume), a command forthe dishwasher 22 (e.g., a cleaning mode or on/off), a command for thegarbage disposal 23 (e.g., a speed or on/off), the refrigerator 24(e.g., select a temperature), a command for the water heater 25, (e.g.,select a temperature), or a command for the water filter 26 (e.g.,activate or deactivate). In addition or in the alternative, data fromexternal device 30 to impact one or more settings of the toilet. In oneexample, weather data from a weather service is used to determine atemperature for the heater of the toilet seat.

Rather than the direct commands, the auxiliary command for anotherdevice may be calculated or determined from sensor data collected at theintelligent toilet 180. There may be a direct correlation between theintelligent toilet 180 and the auxiliary device. The toilet seat maytrigger an action in the auxiliary device. For example, when the usersits on the toilet seat, the fan turns on. In another example, when theuser sits on the toilet seat a media player (e.g., radio, digitalrecording, news) plays and when the user stands from the toilet seat themedia player stops. The position of the user with respect to theintelligent toilet may trigger a specific action in the auxiliarydevice. For example, when the user stands from the toilet seat orotherwise steps away from the intelligent toilet, the faucet turns on ora light guide points toward the sink to encourage handwashing. When thesensor data is stool analysis, the result of the analysis may impact asuggestion for a type of food made at a refrigerator.

The control module 183 may generate alerts from the sensor data. Forexample, when the sensor data indicates a malfunction, an alert is sentto the manufacturer or service provider as the external device 30. Whenthe sensor data relates to urinalysis, an alert may be sent to anemployer or doctor as the external device 30.

When the sensor data describes whether the toilet is in use or not, analert may be sent to another appliance. For example, a television, arefrigerator, or the home hub communication device may provide the alertto the user that the intelligent toilet is in use or not in use. Theoccupancy may also be provided to phone 181. Multiple toilets may beconnected to the network device 14 and server 13. The phone 181 or otherdevices may provide the occupancy toilets or provide an alert when aleast one of the toilets is unoccupied.

Settings data for the auxiliary device may be based on informationreceived at the control module 183 from another of the auxiliarydevices. For example, data collected regarding height from a showersetting may be applied to a toilet seat position, a position of the userinterface on the mirror substrate or a position of a light guide.Configuration data for the user may be based on the identification ofthe user or detection of the user and preferences previously establishedfor the user. Type data for the auxiliary device may be used to apply asetting across a class of devices. For example, the user preferences mayspecify a temperature that is applied to all auxiliary devices with atemperature setting.

In one example, the control module 183 includes the componentsillustrated by FIG. 9. The control module 183 may include acommunication interface configured to send the auxiliary command to theauxiliary device.

The control module 183 may include a speaker configured to provide audiofor status data for the auxiliary device, settings data for theauxiliary device, configuration data for the user, or type data for theauxiliary device. The speaker may be movable. The control module 183 mayengage a positioning mechanism (e.g., stepper motor or solenoid) to movethe speaker toward a user. The degree of movement may depend on the taskperformed by the user, the identity of the user, the height of the user,the preferences of the user. The volume of the speaker may beconfigurable. The control module 183 may set a volume of the speakerbased on the task performed by the user, the identity of the user, theage of the user, the height of the user, the preferences of the user.For example, volume may be set proportional to age. Certain tasks mayinvoke higher volumes. For example, when the shower is running a highervolume is used.

The control module 183 may include a microphone configured to collectaudio (e.g., voice commands) for settings data for the auxiliary device,configuration data for the user, or type data for the auxiliary device.The microphone may be movable. The control module 183 may engage apositioning mechanism (e.g., stepper motor or solenoid) to move themicrophone toward a user. The degree of movement may depend on the taskperformed by the user, the identity of the user, the height of the user,the preferences of the user. The volume of the microphone may beconfigurable.

The voice commands received at the control module 183 may include deviceidentifiers that describe the auxiliary device and device functions tobe performed by the auxiliary device, which are described herein withrespect to various appliances. In addition, the voice commands mayinclude a device identifier for the intelligent toilet, or omitting thedevice identifier may default to the intelligent toilet, and a devicefunction for the intelligent toilet. Example functions performed at theintelligent toilet may include flushing the toilet, activating ordeactivating the seat warmer, activating or deactivating a media player,opening or closing the lid, control of the lights, control of the lightguide, selection of the collected data and selection of the displayeddata.

The control of the lights (e.g., light strip 186) may include the colorof the lights, brightness of the lights, intensity of the lights, orschedule for the lights. The control of the light guide may include anangle or position for the light is determined based on the auxiliarycommand selected for the user. For example, the voice command mayinstruct the light guide to illuminate the basin in response to thevoice command. The selection of the collected data may enable or disableone or more sensors. The selection of the displayed data may enable ordisable the display of external data (e.g., weather) or data receivedfrom auxiliary devices.

Configuration data for the intelligent toilet may include configurationsfor the device that are selected by the user or learned from the user'shabits. Example configurations may be a seat position, default positionsetting for the lid, or interval for the cleaning system. Configurationdata may be specific to the size of the user or handedness (e.g., lefthanded or right handed), which impact how sensor data is recorded (e.g.,handedness may impact the collect of sensor data for usage statistics).Settings data for the intelligent toilet may include settings for thecomponents of the intelligent toilet. Example settings includetemperature for the foot warmer or seat warmer, volume for the mediaplayer, intensity for the light or a delay setting between the time thesensor data is collected and the resulting action is taken (e.g., timebetween the proximity sensor detect a user and the lid is opened).

FIG. 20 illustrates an example control panel 195 for the home hubcommunication devices. Control panel 195 may include phone 181 and/ordedicated remote 185. In some embodiments, control panel 195 is a mobiledevice. The control panel 195 displays multiple inputs or icons to theuser. A selection made on the control panel 195 is encoded in a commandthat may be transmitted directly to the control module 183. Eachselection on the control panel 195 may correspond to a similar voicecommand having a device identifier and a device function. The commandmay be transmitted to the network device 14 and/or server 13, whichrelays the command to the intelligent toilet. The control panel 195 mayadditionally or alternatively control other appliances according to anyof the examples herein. The control panel 195 may be a phone, tablet,watch, or wearable device. The control panel 195 may physically connectto one or more of the bath room or kitchen appliances (i.e., the controlpanel 195 may physically connect (e.g., snap) to one appliance, beremoved, and subsequently be physically connected to another appliance).

Kitchen Appliances

FIG. 21 illustrates a kitchen appliance 210 including a home hubcommunication device, according to an exemplary embodiment. The kitchenappliance 210 includes at least a control module 213, a sensor 214, anda communication interface configured to communication with the networkdevice 14 and the server 13. In one example, the kitchen appliance 210is a kitchen faucet including water inlet pipes 217, a handle 218, and aremovable sprayer 219 with handle grip. The handle 218 controls one ormore valves for on/off and temperature control. The control module 213may electrically connect with sensors and/or valves in the faucetthrough conduit 216 which includes one or more wires or electricalconductors. The sensor 214 may be in the faucet and may be aligned withthe window to detect motion outside of the faucet. The control module213 may control one or more valves to control water flow (e.g., volume,temperature, duration), and the one or more valves may be internal tothe control module 213 or in the faucet itself. The control module 213may include a manual bypass switch 209 for disabling the control module213 and/or electronic control of the faucet. Additional, different, orfewer components may be included.

The kitchen appliance 210 is a water consuming device connected to awater supply through one or more conduits or pipes. Examples include akitchen faucet, a dishwasher, a refrigerator, a water heater, and awater filter. Each of the kitchen appliances is configured to collectdata indicative of a user, communicate the data indicative of the userto another appliance either directly or indirectly, and provide servicesto user based on data indicative of the user that is received from otherappliances.

The sensor data collected by the sensor 214 may describe the water ofthe kitchen appliance 210 or describe the user near the kitchenappliance 210. For detecting the water, the sensor 214 may be located inthe water flow of the kitchen appliance 210. The sensor 214 may be awater quality sensor including pH sensor, oxygen sensor, or anotherchemical sensor. The sensor 214 may be implemented using one or morescreens or filters for detecting particulates in the water. For example,a graphene mesh may include holes of a predetermined size sized fordifferent materials.

The sensor 214 may be a flow sensor, which is a pressure sensor,ultrasonic sensor or light sensor. The light sensor may measure thequantity of water that passes a light beam. The ultrasonic sensorgenerates an ultrasonic wave that travels through the flow of water andis received at a received. Based on the received ultrasonic wave thevolume and/or speed of the flow of water is detected. The sensor 214 maybe paired with two polished surface that reflects the ultrasonic wave orthe light beam on the opposite side of the flow of water and returns theultrasonic wave or the light beam to the sensor 214.

For detecting the user, the sensor 214 may be a proximity sensor (e.g.,infrared sensor) or an image collection device described herein. Forexample, the sensor 214 may be mounted in a neck of a faucet and maydetect a gesture or presence of a hand near the neck of the faucet toactivate or deactivate the flow of water through the faucet.Alternatively or in addition, the sensor 214 may detect the position oridentity of the user as described herein.

The control module 213 may select, based on the analysis of the sensordata, an auxiliary command for an auxiliary device coordinated with thekitchen appliance. The auxiliary command may be selected for aparticular user. The auxiliary command for the auxiliary device is basedon the instruction from the user received at the user interface and thedata displayed at the user interface includes status data for theauxiliary device, settings data for the auxiliary device, configurationdata for the user, or type data for the auxiliary device.

The analysis of the sensor data may determine an instruction receivedvia voice command or user interface. For example, the user may enter acommand for the programmable shower 2 (e.g., select shower sequence orturn on/off shower), a command for the bathtub sensory device 3 (e.g.,select a bath experience), a command for the bathtub level device (e.g.,select a bath level), a command for the intelligent toilet 6 (e.g.,flush or close the lid), a command for the toilet seat 7 (e.g., raise orlower the toilet seat), a command for the sink faucet 8 (e.g., select atemperature or volume), a command for light guides (e.g., a position orangle for the light), a command for the fan (e.g., a fan speed oron/off), a command for kitchen faucet 21 (e.g., select a temperature orvolume), a command for the dishwasher 22 (e.g., a cleaning mode oron/off), a command for the garbage disposal 23 (e.g., a speed oron/off), a command for the refrigerator 24 (e.g., select a temperature),a command for the water heater 25, (e.g., select a temperature), or acommand for the water filter 26 (e.g., activate or deactivate).

Settings data for the auxiliary device may be based on informationreceived at the control module 213 from another of the auxiliarydevices. For example, data collected regarding water temperature from ashower setting or a bathroom sink may be applied to a kitchen sink.Configuration data for the user may be based on the identification ofthe user or detection of the user and preferences previously establishedfor the user. Type data for the auxiliary device may be used to apply asetting across a class of devices. For example, the user preferences mayspecify a temperature that is applied to all auxiliary devices with atemperature setting.

In one example, the control module 213 includes the componentsillustrated by FIG. 9. The control module 213 may include acommunication interface configured to send the auxiliary command to theauxiliary device.

The control module 213 may include a speaker configured to provide audiofor status data for the auxiliary device, settings data for theauxiliary device, configuration data for the user, or type data for theauxiliary device. The speaker may be movable. The control module 213 mayengage a positioning mechanism (e.g., stepper motor or solenoid) to movethe speaker toward a user. The degree of movement may depend on the taskperformed by the user, the identity of the user, the height of the user,the preferences of the user. The volume of the speaker may beconfigurable. The control module 213 may set a volume of the speakerbased on the task performed by the user, the identity of the user, theage of the user, the height of the user, the preferences of the user.For example, volume may be set proportional to age. Certain tasks mayinvoke higher volumes. For example, when the kitchen appliance 210 isrunning a higher volume is used.

The control module 213 may include a microphone configured to collectaudio (e.g., voice commands) for settings data for the auxiliary device,configuration data for the user, or type data for the auxiliary device.The microphone may be movable. The control module 213 may engage apositioning mechanism (e.g., stepper motor or solenoid) to move themicrophone toward a user. The degree of movement may depend on the taskperformed by the user, the identity of the user, the height of the user,the preferences of the user. The volume of the microphone may beconfigurable.

The voice commands received at the control module 213 may include deviceidentifiers that describe the auxiliary device and device functions tobe performed by the auxiliary device, which are described herein withrespect to various appliances. In addition, the voice commands mayinclude a device identifier for the kitchen appliance 210, or omittingthe device identifier may default to the kitchen appliance 210, and adevice function for the kitchen appliance 210. Example functionsperformed at the kitchen appliance 210 may include selection of thetemperature of the water, selection of a water supplement, selection ofa water filter, selection of the displayed data, and control of a lightguide.

The control of the light may include the color of the lights, brightnessof the lights, intensity of the lights, or schedule for the lights. Thecontrol of the light guide may include an angle or position for thelight is determined based on the auxiliary command selected for theuser. For example, the voice command may instruct the light guide toilluminate handwashing in response to the voice command. The selectionof the collected data may enable or disable one or more sensors. Theselection of the displayed data may enable or disable the display ofexternal data (e.g., weather) or data received from auxiliary devices.

The kitchen appliance 210 may select a supplement mode or filtering modebased on sensor data received from another device. The supplement modemay insert a supplement into the water. The supplements may includefluoride, vitamins, nutrients, medicines, or other substances. Thesupplement mode may be activated or modified in response to sensor datacollected at the kitchen appliance 210 or other appliances.

For example, the kitchen appliance 210 may detect the makeup of thewater and select the supplement based on deficiencies in the water. Inanother example, the condition of the user is detected at otherappliances (e.g., the mirror assembly or the intelligent toilet). Thecondition of the user may be malnutrition, which is treated by vitaminsupplements added to the water. The condition of the user may be asickness, which is treated by medicine supplements add to the water.

Intelligent Shower

FIG. 22 illustrates a shower 190 including a home hub communicationdevice, according to an exemplary embodiment. The shower 190 includes ashower enclosure and several shower subsystems (i.e., a water subsystem,an audio subsystem, a steam subsystem, a lighting subsystem, etc.). Eachof the shower subsystems has output devices (e.g., shower outlets, flowcontrol valves, temperature control valves, solenoids associated withthe valves, lighting devices, audio output systems, steam outlets, etc.)configured to provide a user of the shower with an enhanced showeringexperience. The shower 190 may include multiple water dispensers 191,sensors 192, and at least one control panel 195.

The control panel 195 includes an electronic display. The electronicdisplay is configured to display graphical user interfaces for allowinguser control of the various shower subsystems and/or shower outputdevices. A controller is in communication with the electronic displayand causes the graphical user interfaces to be presented via theelectronic display. In various embodiments, the controller may beintegrated with the control panel 195, physically separate from thecontrol panel 195, or partially integrated and partially separate fromthe control panel 195. The control panel 195 may include atouch-sensitive panel overlaying the electronic display (e.g., acapacitive touch screen), manually-operable buttons (e.g., capacitivetouch buttons), and/or other user input devices configured to receiveuser input and provide the user input to the controller. The controlpanel 195 (e.g., via the controller) controls the various components ofthe shower in response to the user inputs (e.g., signals or datarepresenting the user inputs) received at the user input devices.

A shower control system is provided for receiving and processing userinputs, displaying a graphical user interface on the electronic display,and controlling outputs of the various output devices. The showercontrol system advantageously includes software that causes thegeneration and display of intuitive graphical user interfaces forproviding an intuitive and powerful control experience to the user.Settings and combinations of settings may be saved in the shower controlsystem (e.g., a controller of the system) for later playback (e.g.,execution) by a controller of the shower control system. Such playbackor execution causes actuation, adjustment, or another state change ofone or a plurality of the shower output devices.

Shower 190 includes a water subsystem having various output devices(i.e., shower outlets) located within the shower enclosure. For example,shower 190 is shown to include multiple water dispensers 191 including afront showerhead, a left showerhead, a right showerhead, an upper bodyspray, a middle body spray, a lower body spray, side body sprays, ahandshower, and a rainhead. In various embodiments, the water subsystemor set of output devices may include any number or combinations ofoutput devices. For example, in an alternative exemplary embodiment, thewater subsystem may include a central body spray (e.g., a verticalcolumn of shower outlets) in place of upper body spray and middle bodyspray. In another exemplary embodiment, the left showerhead and rightshowerhead may be located on front wall. Shower outlets may be locatedon any of surfaces and may include additional or fewer shower outlets invarious embodiments.

The water subsystem may include one or more analog or digital valves.Valves of the system may be configured to allow for an electronicallycontrolled mixing of hot and cold water. Such mixing can allow controlsystems and methods described herein to achieve or approach certaintarget temperatures. Valves of the system may also be configured toallow for electronically controlled or selected shower outlet waterflow. The electronically controlled valves (e.g., solenoids foractuating the hydraulic valves) are controlled via control signals fromone or more controllers of the shower control systems describedthroughout this disclosure. The valves may be used to independentlycontrol flow volume to each of shower outlets.

In some embodiments, the water subsystem includes multiple differenttemperature control valves (e.g., thermostatic valves). Each temperaturecontrol valve may have a plurality of outlet ports (e.g., three outletports, six outlet ports, etc.). A first temperature control valve maycontrol the temperature of water provided to a first subset of showeroutlets and a second temperature control valve may control thetemperature of water provided to a second subset of shower outlets. Forexample, a first temperature control valve may control the temperatureof water provided to shower outlets, whereas a second temperaturecontrol valve may control the temperature of water provided to othershower outlets. Advantageously, using multiple different temperaturecontrol valves allows the water from different shower outlets to havedifferent temperatures. In other embodiments, a single temperaturecontrol valve is used to control the temperature of water provided tothe various shower outlets. In various embodiments, any number oftemperature control valves may be used to define any number oftemperature zones.

In some embodiments, shower 190 includes a steam subsystem. The steamsubsystem includes steam outlets that receive steam from a steamgenerator in fluid communication with steam outlets. The steam generatoris disposed between, and coupled via conduit (e.g., piping or tubing),to steam outlets and a water supply. The steam generator heats thewater, turning it into steam that is then communicated into showerenclosure through the steam outlets. The steam generator may becontrolled via control signals from one or more controllers of theshower control systems described throughout this disclosure.

In some embodiments, shower 190 includes a music subsystem. The musicsubsystem includes speakers, an amplifier, and a media player. Theamplifier, media player, and other components may be located proximateto or remote from shower enclosure. The music subsystem is configured tocommunicate sound into shower enclosure. The music subsystem (e.g., amedia player thereof) may be controlled via control signals from one ormore controllers of the shower control systems described throughout thisdisclosure.

In some embodiments, shower 190 includes a lighting subsystem. Thelighting subsystem includes one or more lights, such as conventionallight bulbs (e.g., incandescent, LED, fluorescent) or a plurality ofcolored lights configured for use as a lighted rain panel used forchromatherapy. In some embodiments, lights are integrated with rainhead.The lighting subsystem is configured to selectively supply light intoshower enclosure. The lighting subsystem (e.g., particular switches forthe lights, dimmers for the lights, etc.) may be controlled via controlsignals from one or more controllers of the shower control systemsdescribed throughout this disclosure.

In some embodiments, a control panel 195 is configured to receive userinputs for controlling the shower subsystems and for communicatingsettings and status information of the shower subsystems to a user.Control panel 195 generally includes a housing and an electronic display(e.g., an LCD panel). The housing includes various attachment points(e.g., brackets, fasteners, portions for receiving screw heads, etc.)for mounting control panel 195 within shower enclosure. The housing alsoprovides a waterproof casing to protect electronic display andassociated internal electronic components from moisture. Atouch-sensitive panel (e.g., a capacitive touch panel) may also beprovided on the housing for receiving user inputs. A portion of thetouch-sensitive panel may overlay electronic display to provide atouchscreen interface. The electronic display can be caused to displaygraphical user interfaces and to receive user inputs via the touchscreen interface.

Referring now to FIG. 23, a block diagram of a shower control system 200is shown, according to an exemplary embodiment, which is another exampleof a home hub communication device. Shower control system 200 generallyrefers to the electronics involved in processing and communicatingsignals for controlling shower subsystems 230-270 according to userinputs, but may also refer to any of the controlled shower subsystems230-270 or shower output devices themselves. Shower control system 200receives indications to change conditions of the various output devices(e.g., from the user input devices) and acts upon the indications bysending signals to control panels 195, shower subsystems 230-270, and/ordevices/controllers thereof.

Shower control system 200 includes a controller 260 in communicationwith one or more control panels 195. Each of control panels 195 may bedisposed at a different location (e.g., in shower, outside shower, etc.)for facilitating user interaction with shower control system 200 atmultiple different locations. In various embodiments, controller 260 maybe integrated with one or more of control panels 195 or separate fromcontrol panels 195. Controller 260 may receive input from control panels195 (e.g., via communications interface 264) and may control the userinterface outputs provided via electronic display 161. Controller 260processes user inputs received at control panels 195 (e.g., user inputsreceived via a touchscreen, buttons, switches, or other user inputdevices of control panel 195) and provides control outputs to showersubsystems 230-270 based on the user inputs.

Controller 260 communicates with shower subsystems 230-270 and/or thedevices thereof (e.g., shower outlets, speakers, lights, valves, etc.)for controlling the various output devices. For example, controller 260may receive an indication to adjust the temperature of the waterprovided by one or more of shower outlets (e.g., based on user inputreceived at a touch panel interface), and act upon the indication bycausing water with increased temperature to flow through the showeroutlet (e.g., by sending an appropriate control signal to theappropriate mixing valve subsystem). Commands received at controller 260from the microphone or other appliances may include device identifiersthat identify one or more of the shower subsystems 230-270 and devicefunctions for one or more functions performed at the shower subsystems230-270.

Controller 260 may cause electronic display to indicate a target watertemperature, an actual water temperature, and indication of whether theactual water temperature is rising or falling. Controller 260 may causeelectronic display to indicate the requested and completed adjustment intemperature.

In some embodiments, controller 260 is configured to receive signalsfrom control panels 195, steam system 230 including a steam emitter 131,audio system 240, lighting system 250 including a lighting device 151,valve control system 270 (e.g., electronic valves 272-274), and/or othersubsystems or devices of shower control system 200 or external devices(e.g., router 220). Controller 260 processes and acts upon the receivedsignals. Controller 260 may act upon signals received by sending controlsignals to steam system 230, audio system 240, and lighting system 250.Controller 260 may also act upon signals received by sending controlsignals to valve control system 270 (e.g., electronic valves 272-274) orother shower subsystem components. The audio system may include astreaming music module 242, an amplifier 244, and one or more speakers249.

Controller 260 is shown to include a communications interface 264, aprocessor 299, and memory 298. Communications interface 264 may includewired or wireless interfaces (e.g., jacks, antennas, transmitters,receivers, transceivers, wire terminals, etc.) for conducting datacommunications with various systems, devices, or networks. For example,communications interface 264 can include an Ethernet card and port forsending and receiving data via an Ethernet-based communications networkand/or a Wi-Fi transceiver for communicating via a wirelesscommunications network. Communications interface 264 may be configuredto communicate via local area networks (e.g., a home network, a LAN,etc.) or wide area networks (e.g., the Internet, a WAN, etc.).

Still referring to FIG. 23, shower control system 200 is shown toinclude a valve control system 270. According to an exemplaryembodiment, one or more digital valves 272-274 are configured toselectively mix hot and cold water and selectively control water outputto shower outlets 121-129. Each digital valve 272-274 may be arrangedbetween shower outlets (water dispensers 191) and hot and cold watersupplies. In an exemplary embodiment, valves 272-274 include athermostatic mixing component (e.g., for controlling temperature) and/orone or more electrically-actuated solenoids (e.g., for controlling flowvolume). In some embodiments, valve control system 270 includes one ormore sensors for measuring temperature, valve position, and/or waterpressure upstream or downstream of valves 272-274. The sensors may sendsignals with condition information to controller 260, which thenprocesses the signals, and acts upon them.

Valves 272-274 may be electrically operated. In some embodiments,controller 260 controls operation of valves 272-274. Controller 260 mayoperate each of valves 272-274 independently to achieve multipledifferent water temperatures simultaneously. For example, controller 260may cause valve 272 to output water having a first temperature and maycause valve 274 to output water having a second temperature, differentfrom the first temperature.

In some embodiments, the hot and cold water inlets of valves 272-274 arecoupled via a conduit (e.g., piping or tubing) to hot and cold watersupplies, respectively. Valves 272-274 may be actuated by controller 260and/or a separate valve driver circuit. Valves 272-274 may be configuredto control an amount of hot and cold water allowed to pass throughvalves 272-274 to achieve a specified water temperature. Each of valves272-274 may be independently connected to the hot and cold watersupplies and may be operated independently to control the temperature ofthe water provided to a subset of shower outlets (water dispensers 191).

In some embodiments, each of valves 272-274 is connected (e.g., viapiping or tubing) to one or more of shower outlets (water dispensers191). Valves 272-274 may be actuated by controller 260 and/or a separatevalve driver circuit to selectively open and close to control an amountof water (e.g., a flow rate) provided to each of shower outlets (waterdispensers 191). Valve 272 is shown to include three outlet ports andvalve 274 is shown to include six outlet ports. Each of the outlet portsmay be opened and closed independently (e.g., via a solenoid or outletvalve) to independently control the flow rate of water provided to eachof shower outlets (water dispensers 191).

In some embodiments, valves 272-274 do not include outlet valves.Instead, outlet valves may be disposed between valves 272-274 and showeroutlets (water dispensers 191), may be attached directly to showeroutlets 121-129, or may be integral with shower outlets 121-129.According to another exemplary embodiment, valves 272-274 are attacheddirectly to or are integral with shower outlets (water dispensers 191),eliminating the need for outlet valves.

Still referring to FIG. 23, shower control system 200 is shown toinclude a steam system 230, an audio system 240, and a lighting system250. In some embodiments, the control electronics (e.g., controller,microprocessor, data interface) for one or more of subsystems 230-250may be integral with each other and/or combined with controller 260. Forexample, controller 260 may include the control electronics for lightingsystem 250, audio system 240, and/or other subsystems of shower controlsystem 200, thus obviating the need for separate system controlelectronics. In other embodiments, each subsystem may include acontroller and data interface that is configured for receiving signals,processing those signals, and acting upon received signals. Steam system230, audio system 240, and/or lighting system 250 may include sensorsfor detecting conditions of the respective systems, such as temperature,humidity, volume, and luminosity.

The sensor data collected by the sensor 192 may describe the water ofthe shower 190 or describe the user in the shower 190. For detecting thewater, the sensor 192 may be located in the valve control system 270.The sensor 214 may be a water quality sensor including pH sensor, oxygensensor, or another chemical sensor.

The sensor 192 may be a flow sensor, which is a pressure sensor,ultrasonic sensor or light sensor. The light sensor may measure thequantity of water that passes a light beam. The ultrasonic sensorgenerates an ultrasonic wave that travels through the flow of water andis received at a received. Based on the received ultrasonic wave thevolume and/or speed of the flow of water is detected. The sensor 192 maybe paired with two polished surface that reflects the ultrasonic wave orthe light beam on the opposite side of the flow of water and returns theultrasonic wave or the light beam to the sensor 192.

For detecting the user, the sensor 192 may be a proximity sensor (e.g.,infrared sensor) or an image collection device described herein. Forexample, the sensor 192 may be mounted at a faucet or water dispenser191 and may detect a gesture or presence of a hand to activate ordeactivate the flow of water through the faucet. Alternatively or inaddition, the sensor 192 may detect the position or identity of the useras described herein.

One or more sensors 192 may determine volume or weight of the user byimaging or detecting the user from multiple angles. BMI calculationsfrom the volume or weight may be sent to the home hub communicationdevice to be sent to other appliances. The sensors 192 may include othersensors for vital information such as temperature, heart rate, orbreathing rate.

The controller 260 may select, based on the analysis of the sensor data,an auxiliary command for an auxiliary device coordinated with the shower190. The auxiliary command may be selected for a particular user. Theauxiliary command for the auxiliary device is based on the instructionfrom the user received at the user interface and the data displayed atthe user interface includes status data for the auxiliary device,settings data for the auxiliary device, configuration data for the user,or type data for the auxiliary device.

The analysis of the sensor data may determine an instruction received atthe controller 260. For example, the user may enter a command for thebathtub sensory device 3 (e.g., select a bath experience), a command forthe bathtub level device (e.g., select a bath level), a command for theintelligent toilet 6 (e.g., flush or close the lid), a command for thetoilet seat 7 (e.g., raise or lower the toilet seat), a command for thesink faucet 8 (e.g., select a temperature or volume), a command forlight guides (e.g., a position or angle for the light), a command forthe fan (e.g., a fan speed or on/off), a command for kitchen faucet 21(e.g., select a temperature or volume), a command for the dishwasher 22(e.g., a cleaning mode or on/off), a command for the garbage disposal 23(e.g., a speed or on/off), a command for the refrigerator 24 (e.g.,select a temperature), a command for the water heater 25, (e.g., selecta temperature), a command for or the water filter 26 (e.g., activate ordeactivate).

Settings data for the auxiliary device may be based on informationreceived at the controller 260 from another of the auxiliary devices.For example, data collected regarding height from a shower setting maybe applied to a toilet seat position, a position of the user interfaceon the mirror substrate or a position of a light guide. Configurationdata for the user may be based on the identification of the user ordetection of the user and preferences previously established for theuser. Type data for the auxiliary device may be used to apply a settingacross a class of devices. For example, the user preferences may specifya temperature that is applied to all auxiliary devices with atemperature setting.

The controller 260 may control a shower sequence. In one example, theshower sequence involves a combination of a series of pressure settingsfor the water dispensers 191, a series of temperature settings for thewater dispensers 191, a series of pulse settings for the waterdispensers 191, a series of activating or deactivating the waterdispensers 191, or other settings. The shower sequence may be selectedaccording to the user identity, a user preference, or learned over timefor the user based on historical data. The controller 260 accesses theuser configuration and controls the water dispensers 191 to adjustsettings accordingly. The user configuration may include a series ofsettings (e.g., temperature, pressure, pulses) associated with timeintervals. The controller 260 may apply the shower at a first settingfor a first time interval and a second setting for a second timeinterval.

The controller 260 may regulate water usage by controlling the waterdispensers 191. The temperature of the shower, pressure of the shower,or number of active water dispensers 191 may be controlled to encouragethe user to reduce water consumption and stop the shower. The user maybe encouraged to stop the shower when the temperature is too high or notmany water dispensers 191 are working. The controller 260 may access awater usage threshold from the user configuration. The controller 260may calculate the water usage threshold based on local conditionsreceived from the external device 30 (e.g., drought conditions, time ofyear, reservoir levels, or local municipal ordinances or guidelines).

In one example, the controller 260 includes the components illustratedby FIG. 9. The controller 260 may include a communication interfaceconfigured to send the auxiliary command to the auxiliary device.

The controller 260 may be coupled with a speaker configured to provideaudio for status data for the auxiliary device, settings data for theauxiliary device, configuration data for the user, or type data for theauxiliary device. The speaker may be movable. The control module 123 mayengage a positioning mechanism (e.g., stepper motor or solenoid) to movethe speaker as described in other embodiments.

The controller 260 may be coupled with a microphone configured tocollect audio (e.g., voice commands) for settings data for the auxiliarydevice, configuration data for the user, or type data for the auxiliarydevice. The microphone may be movable as described in other embodiments.The volume of the microphone may be configurable.

The voice commands received at the controller 260 may include deviceidentifiers that describe the auxiliary device and device functions tobe performed by the auxiliary device, which are described herein withrespect to various appliances. In addition, the voice commands mayinclude a device identifier for the shower 190. In another example, thelack of the device identifier may default to the shower 190. The voicecommands may include a device function for the shower 190. Examplefunctions performed at the shower 190 may include control of thelighting system 250, control of the steam system 230, control of thevalve control system 270, and control of the audio system 240. Forexample, the voice command may instruct the lighting system 250 toilluminate the shower 190 or a particular water dispenser 191. The voicecommand may activate or deactivate a particular water dispenser 191.

Some systems may include one or more modules which may aggregate varioususer data and information, which can then be used in various ways. Inone example, information may be manually entered or automaticallygathered, such as via an app or web module on a user device (such as acellular phone, computer, tablet, or other user device). Thisinformation may, in some instances, be shared with other third parties.A user may set and/or control the parameters for sharing or exchange ofsaid information with a third party. A user may control a list ofauthorized third parties that can view certain stored information,and/or what information is shared. For example, a user may choose toprovide certain health-related information (such as weight, bloodpressure, heart rate) to a medical provider or physician, but may choosenot to provide certain other information (such as shower preferences,urinalysis, or grooming habits). In some instances, the third party maybe able to communicate directly to the user, such as before, during, orafter viewing the user's information. The third party may be allowed toprovide feedback or additional communications (such as medical advice)to the user through the same web module or app. In some embodiments,users may be able to share certain information with third parties in ananonymous fashion, such that the user's information will not bespecifically traceable to the user, but may provide useful informationto a third party (such as trends over certain demographics of thepopulation). Many other variations are possible.

Intelligent Bathtub

FIG. 24 illustrates a bathtub 600 including a home hub communicationdevice, according to an exemplary embodiment. The bathtub 600 mayinclude a bathtub automatic leveling system, a bathtub automaticmaintaining temperature system, a bubble massage system, a heatedsurface system, an aromatherapy system, a lighted system, a fog or mistgenerating system, chromatherapy, whirlpool, a vibration acousticsystem, or a combination of any of the systems mentioned above. Abathtub 600 may include an integral fill 601, an electronic fill mixervalve 606, a sensor array 602, a drain and drain valve 603, a vibrationacoustic device 605, and a controller 613. Additional, different, orfewer components are included.

The controller 613 is configured to operate the one or more drain valves603 using an electronic drain control algorithm and/or drain valve 603.The controller 613 may automatically operate the drain to empty thebathtub 600 after it detects a person has left the bath through bathsensory devices or/and other devices. The controller 613 can be userdefined to activate or deactivate the autodrain feature and set a usertime delay for autodrain activation. The drain valve 603 may beresponsive to a pulse width modulated signal indicative of an angle ofthe valve or a proportion of fully opening (e.g., 50% open, 80% open)for the valve. The drain valve 603 may include any number ofindependently operating valves and the pulse width modulated signal maycontrol multiple valves. The drain valve may also be operated by DCvoltage or specific current. The drain might have an integral valve andcould be able to open and close electronically by command from thecontroller 613. The drain valve 603 may house some or all of the sensorarray in terms of physical location or in terms of providing sensorytapping points.

The vibration acoustic device 605 may include one or more speakers fordelivery a predetermined sequence of audio and/or vibrations to thebathtub 600. Similar to the vibration acoustic device 605, thecontroller 613 can control other bathtub experiences such aschromatherapy, bask heated surface, bubble massage, and otherexperiences. The controller 613 is configured to generate and provide acontrol signal to the vibration acoustic device 605. The vibrationacoustic device 605 may include a speaker or a transducer to broadcastsound waves that travel through the water of the bathtub 600 and to thebody. The transducers are strategically positioned within the shell ofthe bathtub 600, providing the most beneficial effects to the entirebody, while leaving the bathing surface smooth and uninterrupted. Thecontroller 613 may send the control signal to the vibration acousticdevice 605 according to an audio file such as music. Alternatively, thecontrol panel may be a sequence of pulses corresponding to varyingvibration levels. The vibration acoustic device 605 also may haveBluetooth or Wi-Fi connectivity and may stream music from paired 3^(rd)party devices to play in the bath speakers.

The vibration acoustic device 605 may include speakers mounted on thebathtub 600 in multiple zones. Selection of the zones may be userconfigurations learned based on historical data or directly selected bythe user. The zones may be associate with particular parts of the body.The activation of the zones may vary according with the water level. Thecontroller 613 may determine particular sequences or styles of music orvibration that the user prefers for particular physical conditions.

The sensor array 602 may include a level sensor to detect the waterlevel in the bathtub 600. The level sensor may be a specific gravitysensor having a float. The float may be displaced to a position that isdetected electrically or by position. The force needed to support acolumn of material that is displaced decreases by the weight of theprocess fluid displaced. A force transducer measures the support forceand reports it as an analog signal. The level sensor may include amagnetic sensor including a float. As the float moves up and down acoupled magnet is detected by a magnetic sensor.

The level sensor may include a pressure transducer. The level sensor mayinclude a silicon or piezoelectric diaphragm surface that changes itsresistivity as it expands and contracts due to pressure from a low pointin the bathtub connected to the sensor via tubing. A column of solidmaterial is suspended in the vessel and based on density, sinks to thelowest level to be measured. Alternatively, differential pressuresensors may measure the pressure at the bottom of the bathtub 600 versusa reference level. The level sensor may be an ultrasonic level sensor, alaser level sensor, or a radar sensor.

The level sensor may determine an identity of the user based on the massof the user. The level sensor may use algorithms that may includederivatives to detect when a user enters the bath and/or exits the bathand use that to initiate other connected appliances or experiences—forexample change the lighting or automatically drain after a preset time.The controller 613 may record the water level detected by the levelsensor before the user enters the bath and the water level after theuser enters the bath. The difference between the water levels isproportional to the mass of the user and/or is an identifyingcharacteristic of the user. The controller 613 may determine whether andwhen the user enters the bath based on other sensor data from aproximity sensor. The sensor data from the proximity sensor may alsodetermine how much of the user's body is submerged in the water andadjust the water level calculations accordingly. The level sensorthrough smaller deviations may measure the breath patterns of the useror heart rate of the user, which are health factors for monitoring theuser, and also are identifying characteristics of the user. Thecontroller 613 can use this collected user data to alter the bathexperiences automatically based on predefined or dynamically learningalgorithms.

The sensor array 602 or/and the electronic fill valve 606 may include aflow sensor configured to measure the flow of water into the bathtub600. The sensor array 602 may include an image collection device. Thesensor array 602 may include environment sensors such as a temperaturesensor, humidity sensor, or a pressure sensor. The sensor array 602 mayinclude heart rate sensor. The sensor array 602 may include bloodpressure sensor. The sensor array 602 may include a pressure switch thatactivates the drain valve 603 to open directly acting as an overflowprotection for the bath. The sensor array 602 may include facialrecognition. The sensor array 602 may include proximity sensors todetect the height of the user and may use that as an identifyingcharacteristic of the user. The sensor array 602 may use an objectdetection sensor to detect if there are objects, pets, or people in thebath. The sensor array 602 may include a voice input device ormicrophone. The controller 613 may use algorithms to analyze data fromdifferent sensor array 602 inputs to initiate auxiliary commands. In oneexample, the controller 613 may detect leaks, detect falls, or detectsomeone in distress, and prompt the controller 613 to command actionfrom auxiliary devices, such as call the emergency services.

The controller 613 may select, based on the analysis of the sensor data,an auxiliary command for an auxiliary device coordinated with thebathtub 600. The auxiliary command may be selected for a particularuser. The auxiliary command for the auxiliary device is based on theinstruction from the user received at the user interface and the datadisplayed at the user interface includes status data for the auxiliarydevice, settings data for the auxiliary device, configuration data forthe user, or type data for the auxiliary device.

The analysis of the sensor data may determine an instruction received ata remote (e.g., phone) or other user interface for the bathtub such as acommand for the bathtub sensory device 3 (e.g., select a bathexperience), a command for the bathtub level device (e.g., select a bathlevel) or for another appliance. For example, the user may enter acommand at the bathtub 600 for the programmable shower 2 (e.g., selectshower sequence or turn on/off shower), a command for the intelligenttoilet 6 (e.g., flush or close the lid), a command for the toilet seat 7(e.g., raise or lower the toilet seat), a command for the sink faucet 8(e.g., select a temperature or volume), a command for light guides(e.g., a position or angle for the light), a command for the fan (e.g.,a fan speed or on/off), a command for kitchen faucet 21 (e.g., select atemperature or volume), a command for the dishwasher 22 (e.g., acleaning mode or on/off), a command for the garbage disposal 23 (e.g., aspeed or on/off), a command for the refrigerator 24 (e.g., select atemperature), a command for the water heater 25, (e.g., select atemperature), or a command for the water filter 26 (e.g., activate ordeactivate).

Settings data for the auxiliary device may be based on informationreceived at the controller 613 from another of the auxiliary devices.For example, data collected regarding height from a shower setting maybe applied to a water level (e.g., taller people may be associated withhigh water levels, and wider people may be associated with lower waterlevels). Configuration data for the user may be based on theidentification of the user or detection of the user and preferencespreviously established for the user. Type data for the auxiliary devicemay be used to apply a setting across a class of devices. For example,the user preferences may specify a temperature that is applied to allauxiliary devices with a temperature setting (e.g., temperature settingsfrom the shower are applied by the bathtub 600). In another example, thevibration acoustic device 605 may selected an audio recording or asequence of vibration pulses based on a temporary characteristic of theuser such as mood, stress, exercise level, or temperature as determinedby the mirror assembly 1 or another appliance.

In one example, the controller 613 included the components illustratedby FIG. 9. The controller 613 may include a communication interfaceconfigured to send the auxiliary command to the auxiliary device. Thecontroller 613 may include a speaker configured to provide audio forstatus data for the auxiliary device, settings data for the auxiliarydevice, configuration data for the user, or type data for the auxiliarydevice. The speaker may be movable. The controller 613 may engage apositioning mechanism (e.g., stepper motor or solenoid) according toother embodiments herein. The volume of the speaker may be configurable.The controller 613 may set a volume of the speaker based on the taskperformed by the user, the identity of the user, the age of the user,the height of the user, the preferences of the user. For example, volumemay be set proportional to age. Certain tasks may invoke higher volumes.For example, when the bathtub 600 is running a higher volume is used.

The controller 613 may track the volume and/or size and/or height of theuser may be used to predict the age of the user. Device functions at thebathtub may be limited based on age. For example, children may beprohibited from turning on the water. Device functions at other devicesmay be enabled or disabled based on the age of the user.

The controller 613 may track growth rates or weight loss and generatealerts or report to external device 30 or network device 14 based on thegrowth rates or weight loss. In one example, the network device 14calculated a weight gain or weight loss based on the displacement volumein the bathtub 600. The network device 14 may relay weight data to filefor private transmission to a physician or to coaching/fitness program.

The controller 613 may include a microphone configured to collect audio(e.g., voice commands) for settings data for the auxiliary device,configuration data for the user, or type data for the auxiliary device.The microphone may be movable. The controller 613 may engage apositioning mechanism (e.g., stepper motor or solenoid) to move themicrophone according to embodiments herein. The volume of the microphonemay be configurable.

The voice commands received at the controller 613 may include deviceidentifiers that describe the auxiliary device and device functions tobe performed by the auxiliary device, which are described herein withrespect to various appliances. In addition, the voice commands mayinclude a device identifier for the bathtub 600, or omitting the deviceidentifier may default to the mirror assembly, and a device function forthe bathtub 600. Example functions performed at the bathtub 600 mayinclude water level settings, water temperature settings, bath duration,sensory pattern sequence, media playback, control of the lights, controlof the light guide, or selection of the collected data and selection ofthe displayed data.

The controller 613 may implement a cleaning cycle based on datacollected at the bathtub 600 or at auxiliary devices and receivedthrough the home hub communication device. The cleaning cycle may beimplemented in response to the user being in another room (e.g., theuser is detected by a kitchen appliance). The cleaning cycle may beimplemented in response to the user being positioned near anotherbathroom device. The cleaning cycle may include pre-rinsing the bathtub600 before filing to the water level and/or rising the bathtub 600 afterdraining.

The controller 613 may draw a bath or obtain a specific water level inresponse to a schedule of the user. The schedule may be determined basedon historical data of when the user takes a bath or based on userconfigured preferences such as day and time or based on location such asgeofencing. The auxiliary device may instruct the controller 613 that abath is being requested. The auxiliary device may be an exercise machineor fitness device that communicates the end of a workout as an indicatorthat the user is requesting a bath. A voice command listing the bathtub600 as the device identifier and the bath as the device function may bereceived at the auxiliary device. The controller 613 is configured toprovide the bath without waiting to the user.

Bathroom Cleaning Cycle

A bathroom cleaning device may perform a cleaning cycle for thebathroom. The bathroom cleaning device may be incorporated into the homehub communication device. The bathroom cleaning device may coordinatecleaning cycles of multiple devices including the intelligent toilet 6,the bathtub 600, or other devices. The bathroom cleaning device maycontrol an air filtering system configured to filter aspiratedparticulate (e.g., fecal, urine). The bathroom cleaning device mayinclude a dispenser that propels cleaning solution, or cleaning gas,into the atmosphere of the bathroom. The dispenser may include a safetymechanism that requires a secure seal in the room before dispensing. Theuser may override the dispenser or the bathroom cleaning device frominside or outside of the bathroom. The bathroom cleaning device mayinitiate a cleaning cycle based on data collected by another device(e.g., the cleaning cycle may be initiated based the door to thebathroom being closed).

The bathroom cleaning device may be operated according to sensor datacollected at auxiliary devices. For example, the cleaning cycle may beoperated to agree with the user's typical schedule based on learnedpatterns of operation of one or more of the auxiliary devices (e.g.,intelligent toilet 6, shower 2, bathtub 600, or sink 8). The bathroomcleaning device may create and foster a microbiome between theintelligent toilet 6 and the shower 2 to balance microbial life.

Clothing System

The home hub communication system may also communicate with or include adressing room controller. The dressing room controller may track closetinventory (clean) and pre-laundry (dirty) volume using one or moresensors. For example, the dressing room controller may track laundryinventory using one or more sensors placed in the home (e.g., closet,bathroom, bedroom) that detect individual articles of clothing using aninductive transmitter (e.g., RFID).

The dressing room controller may determine location of specific articlesof clothing, for example, indicate if a desired article is in the closetor in the laundry. The dressing room controller may determine whenlaundry should be done. When a full load, or volume, of a specific typeof clothing (color, type, delicates, etc.) is reached, an alert will bemade through home hub communication device.

The dressing room controller may provide a list of clean clothing to theuser. The list of clean clothing may be available on an interface of oneof the appliances (e.g., mirror assembly 101) for the user to selectand/or visualize clothing to wear.

The dressing room controller may assist a user from shopping from home.One or more appliances may collect sensor data to measure the body ofthe user (e.g., image collection device at mirror assembly 1,displacement volume at bathtub 600, or proximity sensors at otherappliances). The dressing room controller suggests articles of clothingthat fit the user based on the measurements. The dressing roomcontroller may suggest articles of clothing based on the currentinventory of clothing. For example, when a disproportionate amount ofgreen clothing is in the inventory, the dressing room controllersuggests green clothing for purchase.

Replacement Part Generation

Another household appliance in communication with the home hubcommunication device may include a replacement part generator. Thereplacement part generator may include a rapid prototype generator or a3D printer. The replacement part generator is configured to printcomponents for one or more of the appliances. The replacement parts maybe generated through communication directly with the appliance or withexternal device 30 to download dimensions and other specifications forproducing the part.

The replacement part may be generated in response to a remote servicediagnosis of the appliance. For example, a leak detected by a sensor atthe sink may be analyzed by the external device 30 to determine that aseal is broken. The external device 30 instructs the replacement partgenerator to produce a replacement seal. A user interface or the mirrorassembly 1 may display instructions for removing the broken seal andreplacing the broken seal with the newly generated seal. Any part may begenerated and replaced including but not limited to connectors,brackets, covers, caps, handles, knobs, and gaskets.

The replacement part may be generated in response to a request by theuser. For example, the user may request a different style of handle orknob such as the lever on the intelligent toilet 6 or the knobs on thesinks or shower. A user interface or the mirror assembly 1 may displayinstructions for removing the old part and installing the new part.

Recovery Space System

The home hub communication device may also coordinate a recovery spacesystem that helps the user recover from exercise. The sensor data mayinclude images of the user from a camera or other imaging device. Theuser's physical state including muscle strains and usage (e.g., musclemap) are determined from an image analysis. One or more auxiliarydevices are instructed based on the images to help the user recover fromexercise.

The physical state may be communicated to the shower 2 to select amassage pattern or temperature based to encourage recovery. The physicalstate may be communicated to a fitness device to suggest complimentaryworkouts. The fitness device may be a personal fitness device (e.g.,wrist bracelet that tracks movements) and suggests specific exercises oran exercise plan. The fitness devices may be an exercise machine (e.g.,elliptical machine, treadmill, or weight machine). The fitness devicemay generate a targeted recovery experience based on a muscle mapdetermined from analysis of the images.

Medicine Cabinet

The mirror assembly 1 may include a smart medicine cabinet controller incommunication with the home hub communication device and/or the server13 through the network device 14. The smart medicine cabinet controllermay aggregate data collected at other devices such as the appearance ofthe user collected by the mirror assembly 1, the analysis of the waterof the user collected at intelligent toilet 6, and other physical datacollected by the shower 2.

The smart medicine cabinet controller may include a communicationinterface and to communicate with a medical professional system. Themedical professional system may analyze the aggregated data and providea diagnosis or other medical recommendations to the user.

The smart medicine cabinet controller may provide instructions to amedicine generator and/or dispenser. The medicine generator may printmedication such as OTC analgesics, vaccines, antibiotics, daily scripts)via patch or liquid. The medication may be tailored to the particularuser based on sensor data collected at the auxiliary appliance. Themedication may be adjusted according to the age of the user, the weightof the user, or other physical attribute. The medication may be adjustedaccording to the user's profile including allergies or the user's DNA.

Versatile Tiles

A versatile tile may include lights, speakers, sensors, and/or heatersimplemented as flooring. Each of the intelligent bathroom devices isconfigured to collect data indicative of a user, communicate the dataindicative of the user to another intelligent bathroom device eitherdirectly or indirectly, and provide services to user based on dataindicative of the user that is received from other intelligent bathroomdevices.

A versatile tile may include multiple functions that are tied to one ormore of the auxiliary devices. The tiles may be integrated with sensors(e.g., image collection device, proximity sensor, temperature sensor,humidity sensor, VOC sensor, or other sensors) and the sensor data isprovided directly to the home hub communication device on the auxiliarydevice. The sensors may include a pressure sensor configured to detectthe pressure of a user's foot on the tile. The pressure sensor may beactuated by the user to switch between the functions of the tile (e.g.,to switch between light, speaker, etc.). The tiles may include lights(e.g., light guide 9 a). The tiles may include speakers for relayinginformation to the user from one or more of the auxiliary devices. Thetiles may include microphones for receiving voice commands for one ormore of the auxiliary devices. The tiles may include heaters and/orcoolers for adjusting the ambient temperature. The heaters and/orcoolers may be operated in response to temperature data received at oneor more auxiliary devices. The heaters and/or coolers may be operated inresponse to the temperature or other temporary state of the user.

Therapy System

The home hub communication device may also instruct a therapy system.The therapy system may provide therapy to the user in response to thesensor data detected at one or more auxiliary devices. The therapy mayinclude a multisensory experience that combines physical relaxation withvirtual reality, using images, aroma, water, and light. The therapysystem may treat the user with light or with guided imagery andbiofeedback techniques. The therapy system may detect brain health totreat concussion, depression, anxiety, bipolar disorders, addiction,pain management, memory care, or other conditions non-invasively.

Light Guide System

The light guides, including light source 9 a, which may be a lightsource array including one or more directional light sources, mayproject light on one or more other devices to guide the user to thedevice or help illuminate the area near the device. In response to afunction being selected at one of the devices, user selection data, or alocation and/or direction associated with a user selection, istransmitted to the light source 9 a to define a direction for projectingthe projected light guides 9 b. The light guides may be incorporated inone or more of the auxiliary devices.

The light guide may be integrated with a home hub communication device.In one example, a control system for the light guide may include thecomponents illustrated by FIG. 9. As such, the light guide may include acommunication interface configured to connect with the network device 14and/or the server 13 for communication with the auxiliary appliances.The communication interface may receive user data from at least oneappliance, and the user data is indicative of an identity of a user. Thelight guide may include a controller configured to analyze the identityof the user and activate one or more directional light sources inresponse to the identity of the user. The light guide may include amemory including a lookup table for a plurality of users and usersequences for appliances within a predetermined distance of at least onedirection light source in the light source array. The controller isconfigured to access the memory according the identity of the user toselect the one or more activated directional light sources.

The light guide may include one or more sensors in a sensor array. Thesensors may include a proximity sensor configured to detect at least aportion of a use in proximity to the light guide. The sensor may be arelative distance collection device such as a laser scanner. The sensorsmay include any of the environment sensors, microphones, or othersensors described herein. The sensor may be an image collection devicewith a lens such as a camera or an image collection device with a CCD.The sensor may be an audio sensor such as a microphone. The sensor maydetect odors or VOCs.

The light guide may receive user input through voice commands receivedat the microphone or through a user interface at the light guide. Theinstruction from the user may trigger an auxiliary command for theauxiliary device. The data displayed at the user interface includesstatus data for the auxiliary device, settings data for the auxiliarydevice, configuration data for the user, or type data for the auxiliarydevice.

The auxiliary device may be an adjacent device within the same room asthe light guide or less than a predetermined distance to the lightguide. The auxiliary device may be a remote device in a different roomfrom the light guide or greater than a predetermined distance from thelight guide. With respect to the light guide, the adjacent devices mayinclude bathroom devices such as the intelligent mirror 1, theprogrammable shower 2, the bathtub sensory device 3, the bathtub leveldevice, the intelligent toilet 6, the toilet seat 7, the sink faucet 8,light guides, or a fan. With respect to the light guide, the remotedevices may include kitchen devices such as the kitchen faucet 21, thedishwasher 22, the garbage disposal 23, the refrigerator 24, the waterheater 25, and the water filter 26.

The control system for the light guide may select, based on the analysisof the sensor data, an auxiliary command for an auxiliary devicecoordinated with the light guide. The auxiliary command may be selectedfor a particular user. The auxiliary command for the auxiliary device isbased on the instruction from the user received at the light guide andthe data displayed at the user interface includes status data for theauxiliary device, settings data for the auxiliary device, configurationdata for the user, or type data for the auxiliary device.

The analysis of the sensor data may determine an instruction received atthe light guide. For example, the user may enter a command for theprogrammable shower 2 (e.g., select shower sequence or turn on/offshower), a command for the bathtub sensory device 3 (e.g., select a bathexperience), a command for the bathtub level device (e.g., select a bathlevel), a command for the intelligent toilet 6 (e.g., flush or close thelid), a command for the toilet seat 7 (e.g., raise or lower the toiletseat), a command for the sink faucet 8 (e.g., select a temperature orvolume), a command for light guides (e.g., a position or angle for thelight), a command for the fan (e.g., a fan speed or on/off), a commandfor kitchen faucet 21 (e.g., select a temperature or volume), a commandfor the dishwasher 22 (e.g., a cleaning mode or on/off), a command forthe garbage disposal 23 (e.g., a speed or on/off), a command for therefrigerator 24 (e.g., select a temperature), a command for the waterheater 25, (e.g., select a temperature), or a command for the waterfilter 26 (e.g., activate or deactivate).

Settings data for the auxiliary device may be based on informationreceived at the light guide from another of the auxiliary devices. Thatis, the light guide may receive data from one appliance, analyze thedata, and send an instruction to another appliance. For example, datacollected regarding height from a shower setting may be applied to atoilet seat position, a position of the user interface on the mirrorsubstrate or a position of a light guide. Configuration data for theuser may be based on the identification of the user or detection of theuser and preferences previously established for the user. Type data forthe auxiliary device may be used to apply a setting across a class ofdevices. For example, the user preferences may specify a temperaturethat is applied to all auxiliary devices with a temperature setting.

The light guide may include a speaker configured to provide audio forstatus data for the auxiliary device, settings data for the auxiliarydevice, configuration data for the user, or type data for the auxiliarydevice, as described herein. The light guide may include a microphoneconfigured to collect audio (e.g., voice commands) for settings data forthe auxiliary device, configuration data for the user, or type data forthe auxiliary device. The microphone may be movable. The control modulemay engage a positioning mechanism (e.g., stepper motor or solenoid) tomove the microphone and/or the speaker toward a user.

The voice commands received at the light guide may include deviceidentifiers that describe the auxiliary device and device functions tobe performed by the auxiliary device, which are described herein withrespect to various appliances. In addition, the voice commands mayinclude a device identifier for the light guide, or omitting the deviceidentifier may default to the light guide, and a device function for thelight guide. Example functions performed at the light guide may includecontrol of the lights, control of the light guide, selection of thecollected data and selection of the displayed data.

The control of the lights may include the color of the lights,brightness of the lights, intensity of the lights, or schedule for thelights, direction for the lights, hue for the lights, size of the lightbeam or focus for the lights. The control of the light guide may includean angle or position for the light is determined based on the auxiliarycommand selected for the user. For example, the voice command mayinstruct the light guide to illuminate handwashing in response to thevoice command. The selection of the collected data may enable or disableone or more sensors. The selection of the displayed data may enable ordisable the display of external data (e.g., weather) or data receivedfrom auxiliary devices.

The light guide may implement a user sequence including a series oflight positions, which may be defined by angle or direction, duration,intensity, color or other factors. The control system for the lightguide may include a lookup table that associate users with the preferredlighting sequence. The user sequence referenced in the lookup tableincludes a first appliance and a first time and a second appliance and asecond time. The control system may use different lights at differenttimes. For example, the control system may activate a first directionallight source during the first time to illuminate at least a portion ofthe first appliance and activate a second directional light sourceduring the second time to illuminate at least a portion of the secondappliance. For example, a user's sequence may request that after turningon the lights, the sink is illuminated for a set time, then the toiletis illuminated for a set time, then the shower is illuminated for a settime. The user sequence may include a list of appliances and incrementfrom one appliance on the list to the next appliances on the list aftera voice command or other input is received for that appliance. Forexample, the sink is illuminated until the water is turned off, thetoilet is illuminated until the toilet is flushed, or other examples.The user sequence may move from one appliance to the next automatically.

The light guide may receive voice commands for other appliances andilluminate those appliances according to specific tasks. For example,the controller for the light guide may receive a voice command foranother appliance and parse the voice command for at least the deviceidentifier and the device function. The control system may select one ormore lights and a direction for the one or more lights in response tothe device identifier and/or device functions. Device identifiers mayindicate a direction for a particular appliance and device functions mayindicate a direction for a particular portion of the appliance. Thecommunication interface receives task selection data from the at leastone appliance, and the control system is configured to analyze the taskselection data to activate one or more directional light sourcesassociated with a task in response to the task selection data. Exampletasks include any of the device functions described herein such asturning on a sink, showering, opening a storage space, using a mirror,using a toilet, or filling a bathtub.

The light guide may implement the direction, intensity, and duration ofthe light according to user identity. The user identity may bedetermined by sensor data collected at any of the appliances accordingto examples herein. The communication interface of the light guide mayreceive user data from at least one appliance that is indicative of anidentity of a user. The user identity is checked in memory against userpreferences or configurations for the lights. The position of the lightsmay be based on the size of the user, the height of the user, thehandedness of the user, or any other demographic characteristic of theuser. The control system for the light guide analyzes the user data toidentify at least one appliance to be illuminated for the user. Thecontrol of the light guide may also be based on the position of the userand any voice commands or other inputs received by the user. The controlsystem generates a light command for a light source array comprising oneor more directional light sources to illuminate at least a portion ofthe at least one appliance. The direction of the light sources, inaddition to the identity of the user may be based on task selection datafor the identity of the user. The light command for the light sourcearray directs one or more directional light sources to illuminate atleast a portion of the at least one appliance associated with a taskfrom the task selection data.

In addition or in the alternative to the appliance itself, the one ormore direction light sources associated with the task illuminate a pathto the task or to the at least one appliance. The illumination of thetask may be based on the user sequence, which is set by the user orlearned from the user's historical activity. In this way, the taskselection data includes a reminder for the user to perform the task. Forexample, the user sequence may specify that after the toilet is flushed,the sink should be used to wash hands. The reminder may instruct theuser to pick up objects (e.g., clothes from the floor) or to put awayobjects (e.g., return the toothpaste to the drawer. Thus, theillumination of the task may correspond to handwashing. Other examplesmay be a guide to return to bed in the middle of the night or reach fora towel after showering. The sequence of light guides may encouragehabit-building routines for children and those who may have cognitiveissues. The control system may include a clock configured to track timeof day, and the activated one or more directional light sources aremodified according to the time of day.

Water Analysis

Any of the home hub communication devices (e.g., an appliance with anintegrated home hub communication device or a standalone home hubcommunication device), the network device 14, or the server 13 mayinclude a water analysis apparatus. The water analysis apparatus may beintegrated with a water consuming device such as the programmable shower2, the bathtub sensory device 3, the bathtub level device including thedrain 4 and faucet 5, the intelligent toilet 6, the automated sink 8,the kitchen appliance 11, and the water system 12. The water consumingdevice is connected to a water supply or plumbing system.

The water analysis apparatus may include the components illustrated byFIG. 9. As such, the water analysis apparatus may include acommunication interface configured to connect with the network device 14and/or the server 13 for communication with the auxiliary appliances.The communication interface is configured to send a reporting messageindicative of data collected by at least one appliance to the server 13.

The reporting message may include sensor data indicative of the waterquality at one or more of the appliances. The sensors measuring waterquality may include a water quality sensor, a pH sensor, oxygen sensor,or a chemical sensor that detects a specific substance, chemical, orelement, such as mercury, iron, lead,

The sensor may be a biological sensor that detects a type of bacteria orvirus. The biological sensor may detect specific organisms such assalmonella, e. coli, cryptosporidium or listeria.

The sensor may be an image collection device that collects images of theuser for determining an indication of illness, malnutrition,dehydration, or another condition or appearance of the user. The imagesmay be analyzed for a change in color, pigment, or water content.Alternatively, the weight of the user may be monitored to detect signsof dehydration or sickness. The appearance of the user may be monitoredto detect signs of illness, malnutrition, dehydration, or otherconditions.

The reporting message may include device identifiers and timestampsassociated with the collection of the sensor data. The reporting messagemay include geographic locations, which may be collected by a satellitepositioning system, inferred from the IP address of the appliance, orentered directly by the user. The reporting message may include one ormore sensor identifiers for the type of sensor used to collect thesensor data.

The reporting message may be received at another device such as theexternal device 30, the network device 14, or server 13. The networkdevice 14 may analyze the sensor data across multiple appliances in thehousehold. The server 13 or the external device 30 may analyze thesensor data across multiple households or a geographic region. Thereporting message may be analyzed to determine an indication of acondition of water based on the sensor data. The condition of water mayindicate that a foreign substance is in the water.

The external device 30, the network device 14, or server 13 may access adatabase according to the location where the sensor data was collectedor other identifier to identify additional users or appliances in thegeographic area where the sensor data was collected. The neighboringusers may be defined according to water system (i.e., users on the samewater system are considered in the same geographic area for the purposeof distribution the analysis message). An analysis message including theindication of the condition of water is generated in response toanalysis of the sensor data and sent to the users or appliances in thegeographic area. The water analysis apparatus receives the analysismessage from the external device 30, the network device 14, or server13.

The water analysis apparatus may perform geographic analysis on thesensor data from multiple locations. For example, the sensor data isanalyzed for multiple appliances across multiple households or locationsthat are defined according to geographic location. A water conditiondatabase may correlate geographic areas with data collected by homeappliances.

The water analysis apparatus may provide an alert in response to theanalysis message. The alert may warn that a water anomaly has beendetected. The alert may list the foreign substance detected in thewater. The alert may provide instructions (e.g., boil water before use).The alert may include a rating for the water.

The water supply apparatus may monitor the usage of various appliancesto determine whether the alert should be issued. For example, the wateranalysis message may indicate that a foreign substance is present in thewater supply but is only harmful in specific quantities. When the flowsensor, or the aggregate flow sensors in a household, indicative thatthe appliances have used the specific quantities in a predetermined timeinterval, the water supply apparatus presents the alert. The alert maybe based on a water limiting threshold for the volume of water or thetype of water. The water limiting threshold limits the amount of waterprovided by the at least one second appliance or limits a type of waterprovided by the at least one second appliance.

The water analysis apparatus may instruct a filter setting according tothe water analysis message. The filter setting may include filteringmodes that target particular contaminants. A lead filtering mode maytarget removing lead from the water. A bacteria filtering mode maytarget removing bacteria from the water. A specific particulatefiltering mode may target removing a specific particulate from thewater. The filter setting may define a size of filter such as a particletrap opening, a thickness, or a variable path selected in response tothe analysis message.

The water filtering may also be implemented by any faucet, therefrigerator 24, or the water heater 25. In addition, a water additivesystem may be implemented by any faucet, the refrigerator 24, the waterheater 25, the water filter 26, or another water consuming device. Thewater analysis apparatus may instruct a water dispenser configured toselectively insert an additive to the water in response to the analysismessage. The additive comprises a nutritional supplement, a flavor, anantibacterial agent, vaccine, vitamin, mineral, or a medicine.

The water analysis apparatus may perform a water circulation algorithm.The water circulation algorithm may open and close valve in waterconnections between one or more water consuming appliances and the watersupply. Different appliances or different device functions may beapplied to different types of water. Example types of water includepotable water, cleaning water, recycled water, distilled water, orreclaimed filtered water. A set of rules may be applied to the devicefunctions and/or the appliances. In one example, distilled water ispreferred but at least potable water must be used in the waterdispensing of the sink faucets 8 and the refrigerator 24, potable wateris preferred but at least cleaning water may be used in the shower 2,the bathtub 600, or the dishwasher 22, and any type of water includingreclaimed filtered water or runoff recycled water may be used in theintelligent toilet 6.

The water circulation algorithm may be activated by the voice command toroute the water to one or more appliances according to the requirementsof the appliances or device functions performed at the appliance. Forexample, the voice command to flush toilet may instruct the watercirculation algorithm to open a valve that releases water of a firsttype to refill the toilet, and the voice command to turn on the showerinstructs the water circulation algorithm to open a valve that releaseswater of a second type to the shower. The water circulation algorithmmay analyze the voice command to route the water draining from one ormore appliances to a tank for a specific type of water.

The water circulation algorithm may analyze sensor data at one or moreof the appliances (e.g., at the drains) in order to classify the type ofwater and route the water to a tank for storing the water in futureuses. The sensor data may describe the turbidity of the water orparticulates of the water. For example, water in a shower may berecirculated until the water reaches a threshold level. When thethreshold is reached the water circulation algorithm opens a valve torelease the water to drain away from the household.

In one embodiment, the water analysis apparatus may be incorporated inany of the water consuming appliances. The appliances may include atleast one sensor configured to collect data for a water conditionassociated with at least one first appliance and a controller configuredto analyze the data for the water condition and identify or generate aresultant instruction in response to the analysis. The resultantinstruction is sent to at least one second appliance.

Data Logging System

A data logging apparatus may be implemented by the home hubcommunication device, the network device 14, the server 13, or theexternal device 30. The data logging may be integrated with one of theappliances. The data logging apparatus may include the componentsillustrated by FIG. 9. As such, the water analysis apparatus may includea communication interface configured to connect with the network device14 and/or the server 13 for communication with the auxiliary appliances.

The data logging apparatus receives sensor data from multipleappliances. The data logging apparatus may store the sensor data alongwith a sensor identification value, a device identification value,and/or a device function identifier. The data logging apparatus maystore the sensor data with a time stamp. The data logging apparatus mayprovide the logged data from multiple appliances to a requestingappliance. For example, one appliance may request sensor data frommultiple device to identify the user.

The data logging apparatus may analyze the sensor data to identify whenone or more appliances should be serviced, upgraded, or replaced. Forexample, when a water heater temperature drops too quickly, the datalogging apparatus may identify that an element should be replaced. Whenthe water level of a bathtub drops to quickly, the data loggingapparatus may detect a leak or an error with the leveling device, andinform the manufacturer, a service provider, or instruct the user torepair. The data logging apparatus may identify user patterns andrecommend enhancements and recommend hardware upgrades to improveefficiencies in response to the user patterns. For example, therecommended hardware upgrades may include low flow toilets for a hightraffic bathroom or a low flow shower head for heavily used showers. Thedata logging apparatus may recommend the user adjust behaviors such asbathing at certain times, taking shorter showers, or turn off the waterwhen not in use.

The data logging apparatus may provide the logged data to a manufacturerto for monitoring design changes and marketing efforts. In someinstances, the manufacturer may price product under cost or under marketprice to encourage data collection. The data logging apparatus mayprovide the logged data to a third party entity such as a municipalwater company, an insurance provider, a city planner, or another entityaccording to any of the examples described herein.

Interchangeable Speaker

FIG. 25 illustrates a front view of an interchangeable speaker 700,which may be a home hub communication device or an artificialintelligence device as described herein. The term voice controlleddevice encompasses these devices. The interchangeable speaker 700 may beincorporated into the network device 14 in any of the embodimentsdescribed herein. The interchangeable speaker 700 includes a switch 703,which may be a button for turning the speaker 700 on and off, and amicrophone 705 configured to convert sounds to electrical signals. Themicrophone 705 may be omitted in some situations and may be disabled insome scenarios. As illustrated, the switch 703 may include a lightindicator. For example, a light emitting diode behind a transparentshield that indicates whether the speaker is on (light indicator is on)or off (light indicator is off). Alternatively, the light indicator mayindicate a mode, such a low power mode (light indicator is on) or anormal mode (light indicator is off).

The interchangeable speaker 700 may internally include a controllerconfigured to operate the audio function of the host appliance (e.g.,receive verbal inputs and output sound) and also one or more primaryfunctions of a host appliance (e.g., functions outlines herein withrespect to the intelligent mirror 1, the programmable shower 2, thebathtub sensory device 3, the bathtub level device, the intelligenttoilet 6, the toilet seat 7, the sink faucet 8, light source 9 a andlight guides 9 b, the kitchen appliance 11, and the water system 12. Themicrophone 705 is configured to receive one or more voice inputs for thecurrently connected host appliance. For a controller that operates theaudio function of the device, the controller may activate or deactivateone or more microphones 705, provide audio output through the speaker,and connect to the wireless network to receive data that is converted tothe audio output provided through the speaker.

The controller may control additional functions of the host applianceand described in various embodiments herein. The interchangeable speaker700 may be docked to the first appliance 31 and/or the second appliance32, as illustrated in FIG. 4. The interchangeable speaker 700 mayreceive user data from one of the appliances (e.g., at least one waterconsuming device). The user data may describe sensor data collected atthe first appliance 31 and/or the second appliance 32, entry dataentered by the user at the first appliance 31 and/or the secondappliance 32, logged data recorded at the first appliance 31 and/or thesecond appliance 32, or configuration data accessed the first appliance31 and/or the second appliance 32.

The sensor data, or user data, is collected at the first appliance 31and/or the second appliance 32 and describes the spatial position of theuser or gestures of the user. The sensor data may be collected by acamera or image collection device. The entry data can be entered by theuser at the first appliance 31 and/or the second appliance 32. The entrydata may include login data or login credentials (e.g., a username andcertifying information). The entry data may include a selection foroperating the first appliance 31, which may be used to operate thesecond appliance 32. The logged data can be recorded at the firstappliance 31 and/or the second appliance 32 to describe the habits orpreferences of the users over time. The configuration data accessed thefirst appliance 31 and/or the second appliance 32 may relate toparticular features or settings of the appliance.

The switch 703 may control additional systems associated with theinterchangeable speaker 700, and the light indicator may indicate theselection of these additional systems. The switch 703 may switch thecontroller from one or more of a smart speaker mode, a roomcollaboration mode, and a host appliance control mode. Pressing theswitch 703 may select the next of these modes, and the light indicatormay illuminate a first color for the smart speaker mode, a second colorfor the room collaboration mode, and a third color for the hostappliance control mode.

In the smart speaker mode, the interchangeable speaker 700 is configuredto control the audio related functions of the interchangeable speaker700. The audio related functions may include the volume level of thespeaker. The audio related functions may include which microphones areactivated (e.g., the number of microphones, the hardware address of themicrophones, or the pattern of the microphones). The audio relatedfunctions may include receiving data from the wireless network or fromthe host appliance and providing audio in response to the received data.

In the room collaboration mode, the interchangeable speaker 700 isconfigured to receive and provide commands to other appliances otherthan the host appliances. In one example, the commands are received andsent through the wireless network. In another example, the commands areexchanged directly between the interchangeable speaker 700 and the otherappliances (e.g., through Bluetooth, infrared, or other communicationprotocols).

In the host appliance control mode, the interchangeable speaker 700 isconfigured to control other functions of the host appliance. Asdescribed in other examples herein, the interchangeable speaker 700receives data collected by the host appliances and analyzes the dataaccording to user settings or preferences. The interchangeable speaker700 generates user messages in response to the received data and/orgenerates commands for the host appliance in response to the receiveddata.

The interchangeable speaker 700 may include a housing formed by couplingface 700 a to cone 700 b. The face 700 a and the docking portion 700 bmay be welded together, attached by adhesive, or another technique toseal a speaker cone and a printed circuit board (PCB) from externalelements such as water. The interchangeable speaker 700 includes aseries of openings 704 that form a screen in the housing. Behind thescreen, a seal may provide a water tight cavity that allows sound totransfer from the speaker and out of the interchangeable speaker 700.The microphone 705 may be flush with the face 700 a or behind the face700 a.

FIG. 26 illustrates a rear view of an interchangeable speaker 700. Theinterchangeable speaker 700 may have a variety of shapes including coneshaped, a cut-off cone shape, or a rounded cut-off cone shape as shownin FIG. 26.

The docking portion 700 b may include securing mechanism that attachedthe housing to the host appliance. The securing mechanism may include asnap fit connection, a raised track for screwing the interchangeablespeaker 700 into the host device, or a spring biased raised bearing forattaching the interchangeable speaker 700 to the host device.

FIG. 27 illustrates a bathroom setting that includes multiple appliancesor intelligent bathroom devices compatible with the interchangeablespeaker 700. The descriptions provided herein with respect to FIG. 1 maybe applied to similar devices and components in FIG. 27. FIG. 27 alsoillustrates a towel bar 709 mounted to the wall and configured to holdgarments or towels.

The docking portion 700 b may be shaped to fit and interact with avariety of host devices or appliances, such as any of the intelligentbathroom devices described herein. Each of the intelligent bathroomdevices is configured to receive the docking portion 700 b of theinterchangeable speaker 700. While any of the intelligent bathroomdevices may also be configured to receive the interchangeable speaker700, FIG. 27 illustrates that the towel bar 709 is configured to receivethe interchangeable speaker 700, the intelligent mirror 1 is configuredto receive the interchangeable speaker 700, the sink faucet 8 isconfigured to receive the interchangeable speaker 700, the shower wallsor another shower structure is configured to receive the interchangeablespeaker 700, the light guides (e.g., light source 9 a and projectedguides 9 b) are configured to receive the interchangeable speaker 700.In addition, the programmable shower 2, the bathtub sensory device 3,the bathtub level device (e.g., including the drain 4 and faucet 5), andthe intelligent toilet 6. In addition, or in the alternative, thedocking portion 700 b may be shaped to fit and interact with any of theintelligent kitchen devices, as shown in FIG. 2.

The docking portion 700 b may be configured to mate with the intelligentmirror 1 through, for example, the control module housing 122 asillustrated in FIGS. 14A and 14B. The docking portion 700 b may beconfigured to mate with the programmable shower 2, for example, atcontrol module 195, as illustrated in FIG. 22. The docking portion 700 bmay be configured to mate with the bathtub sensory device 3 or thebathtub level device through a housing connected to bathtub 600, asillustrated by FIG. 24. The docking portion 700 b may be configured tomate with the intelligent toilet 6 through control module 157, asillustrated in FIG. 17, or through control module 183, as illustrated inFIG. 18. The docking portion 700 b may be configured to mate with thesink faucet 8, through control module 213, as shown by FIG. 21. Thedocking portion 700 b may be configured to mate with the housing of thelight guides 9 a. The microphone 705 may be disabled when theinterchangeable speaker 700 is docked with certain intelligent bathroomdevices, such as the shower 2 or the intelligent toilet 6.

The interchangeable speaker 700 may be moved by the user from applianceto appliance. The transferability and interchangeability of theinterchangeable speaker 700 provides multiple advantages.

First, in many examples the intelligent bathroom devices and/orintelligent kitchen devices have many redundant parts. For example,consider a bathroom with a separate speaker, microphones, and controllerfor each of a combination of the intelligent mirror 1, the programmableshower 2, the bathtub sensory device 3, the bathtub level device (e.g.,including the drain 4 and faucet 5), the intelligent toilet 6, the sinkfaucet 8, and the light guides (e.g., light source 9 a and projectedguides 9 b). The interchangeable speaker 700 reduces these redundantparts as multiple devices may use the same speaker, microphone, andcontroller as it is moved from host device to host device.

In some examples, the host devices may have permanent controllers andsome of the functionality of the host devices is handled by theinterchangeable speaker 700. The interchangeable speaker 700 may collectdata at one appliance and apply that data at another appliance.

Second, the operation of multiple coordinating artificial intelligencedevices may be coordinated and made more efficient by theinterchangeable speaker 700. With a device (e.g., the device that iscurrently docked with the interchangeable speaker 700) designated aslistener, the process of associating voice commands with their intendedrecipient is simplified. In other words, in a room with multiple deviceswith microphones listening for commands, the commands may be difficultto interpret.

The interchangeable speaker 700 may receive sensor data from one or moreof the intelligent bathroom devices. In some examples, the sensor datamay be indicative of the user identify or movement of the user. Thesensor data may describe the spatial position of the user. For example,the sensor data may be collected by a motion sensor or a proximitysensor. The sensor data may include three-dimensional coordinates of thespatial position of the user of a portion of the user (e.g., feet,hands, or body). The sensor data may be collected by an eye gaze sensorthat determines a line of sight or angular orientation of the face ofthe user. The sensor data may describe a gesture or movement made by theuser. The sensor data may be collected by a camera or image collectiondevice. The sensor data may describe the state of the body of the userusing another characteristic such as heart rate, temperature, thepresence of sweat, or odor. The user data may describe a condition ofthe user. Example conditions of the user includes a temperature, a heartrate, a height, a weight, a drowsiness, a complexion, a malnutrition, ahydration, or other conditions.

The interchangeable speaker 700 may collect any of these types of datatypes. The interchangeable speaker 700 may include at least one sensorconfigured to detect an ambient environment for the currently connectedhost appliance or user activity associated with the for the currentlyconnected host appliance. The interchangeable speaker 700 may alsoreceive entry data from a user such as login data or login credentials.The entry data may include a selection for operating any of the hostappliances, or other appliances.

The interchangeable speaker 700 may access configuration data based onthe sensor data. The selection of the configuration data may depend onwhere or what type of host appliance the interchangeable speaker 700 isdocked to. For example, a particular user may have a preferred volumesetting or use a mute setting to disable the speaker is certainsituations.

Through first docking with one of the intelligent bathroom devices andsubsequently docking with another one of the intelligent bathroomdevices, the interchangeable speaker 700 may transfer data (e.g., sensordata) between the intelligent bathroom devices. The interchangeablespeaker 700 may collect data from multiple intelligent bathroom devicesand different times and log the data. The data from multiple intelligentbathroom devices may then be analyzed at the interchangeable speaker 700or sent to the server 13 for analysis.

FIGS. 28A, 28B and 28C illustrate a docking detection system for theinterchangeable speaker. The cone portion 700 b of the housing of theinterchangeable speaker 700 is an example a dock configured to connectthe housing to any of the host appliances.

FIG. 28A illustrates the interchangeable speaker 700 including multipleopenings 710, which are contact points. FIG. 28B illustrates a hostappliance 713 including a docking receptacle 712 shaped to fit theinterchangeable speaker 700. The docking receptacle 712 includes one ormore protrusions 711, which are contact points. The multiple openings710 of the interchangeable speaker 700 may be associated with differentappliances for mounting the interchangeable speaker 700.

The contact points for openings 710 and the contacts points forprotrusions 711 provide an electrical connection circuit that connectsthe interchangeable speaker 700 to the host appliance 713. Theelectrical connection circuit allows the interchangeable speaker 700 todetect the host appliance 713, or allows the host appliance 713 todetect the interchangeable speaker 700, or both. The detecting device(either the host appliance 713 or the interchangeable speaker 700)detects the detected device (the other of the host appliance 713 or theinterchangeable speaker 700) and generates data or a signal thatindicates that docking has occurred.

The different openings 710 may correspond to different host appliances713 or types of host appliances. For example, one opening 710 may bealigned with protrusion 711 for the mirror 1, another opening 710 may bealigned with protrusion 711 for the shower 2, and another opening 710may be aligned with protrusion 711 for the sink. A circuit internal tothe interchangeable speaker 700 may include resistors, other circuitelements, or circuit paths that are energized differently depending onwhere the protrusion 711 makes contact with openings 710.

Alternatively, only one opening 710 may be used, and the protrusions 711are located in similar positions for the host appliances 713, butanother technique is used to identify the host appliance 713. In oneexample, the signal may have different voltage levels that correspondsto different host appliances. In another example, the signal may varyaccording to a resistance in the detected device. The controller of theinterchangeable speaker 700 may detect the voltage level or resistancevalue to identify which host appliance is docked with theinterchangeable speaker 700. The controller may access a set of commandsor a mode of operation associated with the detect host appliance.

In another example, the communication in the electrical path between theopening 710 and the protrusion 711 may include data that identifies thedocking between the devices or the identity of one of the devices (e.g.,identity of the host appliance). The contact points for openings 710 andthe contacts points for protrusions 711 provide coupling mechanismconfigured to receive an identification value indicative of dockingbetween the voice controlled device and a currently connected hostappliance. The coupling mechanism may include a first electrical contactthrough the opening 710 at a surface of the housing. The firstelectrical contact is operable to transfer the identification valuethrough a second electrical contact at the protrusions 711 of thecurrently connected host appliance.

The identification value for the docking of the interchangeable speaker700 to the host appliance may also be determined based on a user input.For example, the user may press button 703 (e.g., press and hold) toindicate that the interchangeable speaker 700 has been docked. Inresponse any of the coupling mechanisms may be used to determine theidentity of the host appliance. In addition, the user input may includethe identification value (e.g., through a menu selection). A voicecommand may also indicate the identity of the host appliance (e.g., dockspeaker to mirror).

The identification value for the docking of the interchangeable speaker700 to the host appliance may also be determined based on communicationbetween the interchangeable speaker 700 and the host appliance throughthe wireless network and/or server 13. The controller 804 may identifythat the interchangeable speaker 700 is docked based on a connection tothe wireless network (i.e., when the interchangeable speaker 700 isconnected to the same network as the host appliance). The identificationvalue for the docking of the interchangeable speaker 700 to the hostappliance may also be determined based on GPS sensors. Theidentification value for the docking of the interchangeable speaker 700to the host appliance may also be determined based on machine learninganalysis of other habits of the user.

Commands may be provided to the host appliances based on theidentification value that is indicative of the host appliance and/orbased on one or more voice inputs received at the microphone 705. Acommand is provided based on the one or more voice inputs and theidentification value. In some examples, the command is generated at theinterchangeable speaker 700 to instruct the speaker to produce sounds inresponse to the one or more voice inputs and the identification value.In some examples, the command is generated at the interchangeablespeaker 700 to instruct a display to provide information regarding thecommand or operation of the host appliance. In some examples, thecommand is generated at the interchangeable speaker 700 to instruct thehost appliance to control a host appliance such as turn on a function ofthe host appliance (e.g., turn on water, turn on light), change asetting (e.g., water temperature, water flow rate or volume, or a lightintensity), or provide information regarding the operation of the hostappliance. The interchangeable speaker 700 may provide the informationthrough the display or speaker. Thus, the operation of the speakerincluding the sounds produced by the speaker are adjusted based on theidentification value.

The interchangeable speaker 700 or the host appliance may also includeone or more light guides or illumination devices configured to producelight in response to the one or more voice inputs and the identificationvalue. The light guides may include light source 9 a and projectedguides 9 b but may also be included as light sources on any of the hostappliances. The light guides may project light on one or more otherdevices to guide the user to the device or help illuminate the area nearthe device. The light guide from one of the host appliances mayilluminate another host appliance to guide the use between appliances.In response to a function being selected at one of the devices, userselection data, or a location and/or direction associated with a userselection, is transmitted to the light guide to define a direction forprojecting the projected light guides. The status of the light guidesmay be transmitted to and displayed by the display of theinterchangeable speaker 700. In one alternative, the light guides areimplemented as versatile tiles or tech tiles. The versatile tiles mayinclude lights, speakers, sensors, and/or heaters implemented asflooring.

FIGS. 29A and 29B illustrate an inductive coupling system for theinterchangeable speaker 700. The interchangeable speaker 700 may includean induction circuit 720 including an inductive coil that generates amagnetic field including the identification value or receives a magneticfield including the identification value. The host appliance may includean induction circuit 722 arranged inside the docking receptacle 712 ornear the docking receptacle. The induction circuit 722 may include aninductive coil that generates a magnetic field including theidentification value or receives a magnetic field including theidentification value. The identification value may be modulated on themagnetic field. In other words, the inductive coil has a carrierfrequency or amplitude that is modified according to an identificationvalue from the induction circuit. The identification value correspondsto the type of host appliance, the specific identity of the hostappliance, the type of interchangeable speaker 700, or the specificidentity of the interchangeable speaker 700.

The inductive coupling system may include a coupling mechanism based onthe cooperative relationship of the induction circuit 722 including afirst induction coil attached to the voice controlled device. The firstinduction coil is operable to transfer the identification value throughthe magnetic wave to a to the induction circuit 720 including theinduction coil attached to the currently connected host appliance. Theinductive coupling system may include a coupling mechanism based on thecooperative relationship of the induction circuit 720 including a firstinduction coil attached to the voice controlled device. The firstinduction coil is operable to transfer the identification value throughthe magnetic wave to a to the induction circuit 722 including theinduction coil attached to the currently connected host appliance.

FIGS. 30A, 30B and 30C illustrate a magnetic coupling system for theinterchangeable speaker. The interchangeable speaker 700 may include oneor more magnets 730 arranged at the surface of the housing or under thehousing. The host appliance may include one or more magnets 734 at asurface of docking receptacle 721 or behind the surface of the dockingreceptacle 721. The magnets 730 of the interchangeable speaker 700interact with the magnets 734 of the host appliance.

The magnets may provide a physical coupling or support for theinterchangeable speaker 700 that is docked in the host appliance. Eitheror both of the magnets 730 of and the magnets 734 may supply a magneticforce to the other of the magnets 730 of and the magnets 734, whichtends to provide a docking force between the interchangeable speaker 700and the host appliance. In other words, the magnets 730 of and themagnets 734 may hold the interchangeable speaker 700 to the hostappliance through magnetic force.

The magnets 730 of and the magnets 734 may be arranged in variouspatterns around the interchangeable speaker 700 and the dockingreceptacle 721, respectively. The arrangement of the magnets may ensurethat the interchangeable speaker 700 is docked in the correct, uprightposition. For example, their may be only one way for the magnets 730 ofand the magnets 734 to align to provide the docking force.Alternatively, a physical guide may provide the alignment between theinterchangeable speaker 700 and the docking receptacle 721. For example,a guide rail 732 of the interchangeable speaker 700 may fit in a guidegroove 735 of the docking receptacle to ensure that the interchangeablespeaker 700 is docked in the correct, upright position. In otherexamples, the interchangeable speaker 700 may be docked in a variety ofangles, with the magnets 730 of and the magnets 734 have multiple radialalignments possible.

The magnets 730 of and the magnets 734 may provide identification of thehost appliance to the interchangeable speaker 700 or identification ofthe interchangeable speaker 700 to the host appliance. The magnets 730of and the magnets 734 may be a coupling mechanism that physicallycouples and/or magnetically couples the at least a first magnet ofmagnets 730 of the voice controlled device to at least a second magnetof the magnets 734 attached to the host appliance. The identificationmay be provided through different arrangement of magnets in differentdocking receptacles associated with different host appliances.

The controller or control circuit of the interchangeable speaker 700 mayinclude one or more magnetic sensors or magnetically completed circuitpaths that correspond to the magnets 730. When the magnets 730 arecoupled with magnets 734, the magnetically completed circuit paths arecompleted or the magnetic sensors are triggered to detect the docking ofthe interchangeable speaker 700. The controller may determine theidentification value provided through the magnetic coupling.

The magnetic coupling system may include a coupling mechanism based onthe cooperative relationship of the magnet 730 and the magnet 734. Thecontroller is operable to detect the identification value through thedetect of the relationship between the magnet 730 and the magnet 734.

FIGS. 30B and 30C illustrate two example arrangement of magnets 734 intwo different types of host appliances. In FIG. 30B, the magnet 734 isarranged approximately at angle a₁ (e.g., 60 degrees) counter-clockwisefrom the guide groove 735. In FIG. 30C, the magnet 734 is arrangedapproximately at angle a₂ (e.g., 45 degrees) counter-clockwise from theguide groove 735. Host appliances may include multiple magnets inpredetermined patterns in different radial positions as well asdifferent lengthwise positions along the housing of the interchangeablespeaker 700.

When the interchangeable speaker 700 is docked with the host appliance,different magnets 730 of the interchangeable speaker 700 are coupleddepending on whether the host appliance in FIG. 30B with one magneticposition is docked with the interchangeable speaker 700, or the hostappliance in FIG. 30C with another magnetic position is docked with theinterchangeable speaker 700. The controller is configured to determinethe magnetic position based on the magnetically completed circuit pathsthat are completed or the magnetic sensors that are triggered. Thecontroller may access a magnetic lookup table according to the magneticposition. The magnetic lookup table associated magnetic positions (e.g.,160 degrees, 45 degrees, 60 degrees in combination with 90 degrees) thatcorrespond to identification values that for example, correspond todifferent types of host appliances as described herein.

The interchangeable speaker 700 may include a command lookup table oraccess such a command lookup table from the server 13 or the hostappliance. The command lookup table may be queried with the type of hostappliance based. The command lookup table may return a set of commandsavailable for the host appliance, or values for default or preferredsettings of the host appliance.

In other words, the controller of the interchangeable speaker 700 mayquery the command lookup table using the identification value for thehost appliance. The command lookup table may return a set of commandsfor the host appliance, which may be the commands available for the typeof host appliance or the commands available to the current user. Becauseless than all possible commands may be available, the result from thecommand lookup table may be a reduction of the possible commands. Withfewer commands available, the voice commands from the user may be moreeasily interpreted.

The controller may adjust the output of the interchangeable speaker 700based on the identification of the host appliance. Examples includeadjustment of the sound volume and light intensity of theinterchangeable speaker 700. Certain host appliances may produce noisesthat interfere with the sound output of the interchangeable speaker 700.In addition, host appliances may be in a location that is a fartherdistance from the user that impedes sound travel. The interchangeablespeaker 700 may adjust the volume of the sound according to theidentification value for the host appliance. Similarly, host appliancesmay be in darker environments or farther distances from the user thatinterferes with the light output of the interchangeable speaker 700. Theinterchangeable speaker 700 may adjust the light intensity of a displayor a light guide according to the identification value for the hostappliance. As described in more detail below, the microphones andanalysis of the interchangeable speaker 700 may be adjusted according tothe host appliances or appliances in the vicinity of the interchangeablespeaker 700. The controller may adjust the voice inputs based on theidentification of the host appliance.

FIG. 31 illustrates a flow chart for the interchangeable speaker 700.The method may be performed by the interchangeable speaker 700 as awhole, or specific aspects of the method may be performed by thecontroller of the interchangeable speaker 700 or the coupling device orcircuitry of the interchangeable speaker 700. Additional, different, orfewer acts may be included.

At act S301, the housing of the voice controlled device (e.g.,interchangeable speaker 700) is docked to a host appliance. The dockingmay include one or more guiding mechanisms or locking mechanism thatbring the voice controlled device in contact with and fixedly supportedby the host appliance.

At act S303, the voice controlled device receives an identificationvalue indicative of the docking between the voice controlled device andthe host appliance. The identification value may be included in a datamessage that is transferred wirelessly. The identification value may bedetermined based on inductive coupling or magnetic coupling between thevoice controlled device and the host appliance. Act S303 may alsoinclude magnetically coupling a first magnet attached to the voicecontrolled device to a second magnet attached to the host appliance,such that the identification value is provided through the magneticcoupling and a force for docking the voice controlled device to the hostappliance is provided through the magnetic coupling. Act S303 may alsoinclude energizing a first induction coil attached to the voicecontrolled device to transfer the identification value through amagnetic wave to a second induction coil attached to the host appliance.Act S303 may also include electrically coupling a first electricalcontact at a surface of the housing to transfer the identification valuethrough a second electrical contact of the host appliance.

At act S305, the voice controlled device receives one or more voiceinputs for the host appliance. The voice inputs may be sounds producedby a user providing instruction to the host appliance or the voicecontrolled device.

At act S307, the voice controlled device provides a command based on theone or more voice inputs and the identification value. In addition, thevoice controlled device may provide audio output from a speaker of thevoice controlled device in response to the one or more voice inputs andthe identification value. The audio output content may depend on theidentification value. That is, the voice controlled device providesdifferent audio output when the host appliance is a mirror rather than atoilet. The voice controlled device may adjust the audio output producedby the speaker are adjusted in volume, pitch, or speed based on theidentification value. Different host appliances may be associated withdifferent volumes or other sounds characteristics so that the audiooutput is easier to hear or the host appliance as a signature sound(e.g., different style of voice for different host appliance). The styleof voice may be gender or famous voices. A different famous voice may beapplied to different host appliances. In addition, the style of voicemay be language. The language may be adjusted according to the location(e.g., country) of the host appliance.

Filtering Sound for Intelligent Bathroom Devices

The home hub communication devices and well as the interchangeablespeaker 700 in preceding sections may be configured to adjust theanalysis of voice inputs as a function of the environment of the homehub communication device, the appliances in the vicinity of the home hubcommunication device, or the operation state of the appliances in thevicinity of the home hub communication device.

FIG. 32 is a flow chart for sound filtering for a voice controlleddevice, which may include any of the home hub communication systems orintelligent bathroom devices described herein. The method may beperformed by the voice controlled device as a whole, or specific aspectsof the method may be performed by the server 13. Additional, different,or fewer acts may be included.

At act S401, the voice controlled device identifies an interferingsound. The interfering sound may be a sound caused by water usage of theintelligent bathroom device. The interfering sound may be the sound ofwater spraying and hitting surfaces in the shower, the sound of a toiletflushing and/or filling the tank, or the sound of water flowing from afaucet or shower head. The interfering sound may be non-water basedsounds such as operation of a fan or a heater. Many other sounds arepossible.

The identification of the interfering sound may be made in a variety oftechniques. The interfering sound may be detected directly through amicrophone. The interfering sound may be determined based on thesettings of the appliance producing the sound. That is, the voicecontrolled device may have instructed the shower to turn on or be aware(such as through communication with another interconnected device) thatan instruction was sent to the shower to turn on, and determine that theshower is producing the interfering sound based on the instruction. Thevoice controlled device may determine the presence of the interferingsound based on another factor or combination of factors, such as time ofday, humidity detected in the air, detection of a change incharacteristic of a device (such as detection of water flowing through asystem or detection of a circuit or relay changing states).

At act S403, the voice controlled device determines a characteristic ofthe interfering sound. The characteristic may be the volume or amplitudeof the sound waves. The characteristic may include the frequency orspectrum of the sound waves. The voice controlled device, or anotherdevice in communication with the voice controlled device, may includesensors and supporting circuitry to detect the amplitude of the outputof the interfering sound and/or the frequency spectrum of theinterfering sound. The characteristic may be the types of sound (e.g.,shower, toilet, or faucet). In addition, the voice controlled device mayinclude a temporal characteristic that describes when the interferingnoise is expected to start or end.

At act S405, the voice controlled device receives a voice input. Thevoice input may be words spoken by the user to the voice controlleddevice and in the vicinity of the appliance producing the interferingnoise. In some examples, the user may be closer to the applianceproducing the interfering noise than to the voice controlled device. Forexample, a user may be providing the voice input from inside the showerto the voice controlled device at the mirror across the bathroom. Inother instances, the user may be near the voice controlled device, butthe interfering sound may normally cause disruption in the ability ofthe voice controlled device to differentiate a voice from the noise.

At act S407, the voice controlled device adjusts the voice input basedon the characteristic of the interfering sound. The voice controlleddevice may apply a software or hardware based filter based on theamplitude of the output of the interfering sound and/or the frequencyspectrum of the interfering sound. The voice controlled device may applyequalizer settings based on the amplitude of the output of theinterfering sound and/or the frequency spectrum of the interferingsound. For example, the voice controlled device may amplify or attenuatefrequency bands of the voice input in response to the detectedcharacteristics of the interfering sound. The adjustment of the voiceinput may vary in real time. The interfering sounds may change over timein intensity and/or frequency. The voice controlled device may detectthese changes and adjust the voice inputs accordingly. The voicecontrolled device may perform a noise cancellation algorithm or logic toreduce or eliminate the interfering noise.

The voice controlled device may apply a filter according to the soundcharacteristic. The voice controlled device may store in memory multiplefilters. Each of the appliances or intelligent bathroom devices may beassigned a filter according to the sound that the device makes. Each ofthe appliances or intelligent bathroom devices may be assigned a filterfor a specific user according to the sounds that the user makes with theparticular device.

Because the voice input has been adjusted to account for the interferingsound, the voice controlled device may perform voice recognition at ahigher confidence level. That is, voice commands for the user may bemore easily interpreted even in the presence of the interfering sound.Alternatively, the voice controlled device may record the voice inputsand forward sound files or messages to the server 13, and the server 13can more easily interpret the voice commands even in the presence of theinterfering sound.

The server 13 or the voice controlled device may also store filters orvoice input modification data for different types of devices that are indifferent environments. When a voice input is received, the voice inputmodification data is accessed for a current environment of the voicecontrolled device. The current environment may depend on the usage ofany of the appliances or a setting of any of the appliances.

For example, the server 13 or the voice controlled device may receivedata that indicates that a particular appliance (e.g., a shower) isbeing used. The home hub communication devices, or the interchangeablespeaker 700, that is in the vicinity or the same room as the particularappliance (even if it is not at the particular appliance) may modifysubsequent voice inputs based on the usage of the particular appliance.For example, a sensor at the valve of the shower 2 or the lever of thetoilet 6 may report that the appliance is being used, and accordinglythere will be noise. The server 13 or the voice controlled device mayaccess the filter for one of these scenarios, and subsequently (for apredetermined time) modify voice inputs that are received through themicrophone of the voice controlled device.

In some alternative methods, the voice controlled device may, before,during or after determining an interfering sound, adjust listeningbehavior or requirements. For example, a voice controlled device mayperform any of the filtering and/or cancellation techniques with allsounds received at the voice controlled device automatically andimmediately upon starting at S401, thereby ultimately performing stepS407 right after determining of potential interfering noise. Many othervariations are possible.

In some systems, control and operation of an interchangeable speaker 700may vary as a result of determination of an interfering noise (such asat S401). In some systems, output characteristics (such as amplitudeand/or frequency) of sound waves emitted from the speaker 700 may bemodified to accommodate different sound characteristics present in thesetting. For example, when the system detects the operation of a fan,the speaker 700 output may be modified to increase the amplitude ofsound within various frequency spectrums to cancel undesirable noiseinterference, or accentuate desirable frequencies of sound. Many othervariations are possible.

Power Modes

FIG. 33 illustrates a microphone array for an artificial intelligencedevice 800. The artificial intelligence device 800 may be theinterchangeable speaker 700, or included in any of the voice controlleddevices or intelligent bathroom devices described herein, or networkdevice 14. The artificial intelligence device 800 includes an array ofmicrophones 801 and a controller. The artificial intelligence device 800may optionally include a least one sensor 805 and may optionally includeone or more user inputs (e.g., button 803). In some embodiments, atleast one sensor 805 is external to the speaker or artificialintelligence device 800, for example located in any of the appliances orintelligent bathroom devices described herein. Additional, different, orfewer components may be included.

FIG. 34 illustrates a microphone array 801 across multiple appliances.The devices may be anyone of the appliances (e.g., intelligent bathroomdevices or intelligent kitchen devices) described herein. Each, some, orall of the appliances may include one or more microphones 801.Additional, different or fewer components may be included.

The array of microphones 801 are configured to receive voice inputs fromone or more users for the operation of the artificial intelligencedevice 800 or other devices. In some smart speaker configurations, themicrophones may be “always listening” for voice commands. The termalways listening means that the microphones are powered and can detectvoice commands at any time. The artificial intelligence device 800 maybe powered only by battery, or in some configurations or dockingarrangements, may be powered only by battery. With microphones that arealways listening, battery storage may become problematic because poweris always being consumed. The following embodiments provide techniquesfor the artificial intelligence device 800 that reduce batteryconsumption and improve the duration between charges for a batteryoperated artificial intelligence device 800. The artificial intelligencedevice 800 may be docked or associated with a first appliance, and theartificial intelligence device 800 may be woken up (mode change) bysensor data from the operation of a second appliance and/or detected bythe second appliance.

The artificial intelligence device 800 may include a communicationinterface 806 that is configured to receive sensor data from an externalappliance. The external appliance may be any of the appliances (e.g.,intelligent bathroom device, intelligent kitchen devices, or hostappliances) described herein. The external appliance may include asensor 805, or be associated with the sensor 805. The sensor datadescribes an environment of the artificial intelligence device 800. Thesensor data describes one or more events, users, or activity in theenvironment of the artificial intelligence device 800 that indicates thelikelihood that the artificial intelligence device 800 will soon receivevoice commands.

Examples of the sensor 805 include a flow sensor, an inertial sensor, ahumidity, a pressure sensor, a light sensor, a proximity sensor, or agesture sensor.

The flow sensor may detect a water flow in the external appliance. Theflow sensor may measure water flowing through a pipe using a pressuresensor, ultrasonic sensor or light sensor. The light sensor may measurethe quantity of water that passes a light beam. The ultrasonic sensorgenerates an ultrasonic wave that travels through the flow of water andis received at a received. Based on the received ultrasonic wave thevolume and/or speed of the flow of water is detected. Alternatively, theflow sensor may determine whether a flow of water in the externalappliance is on or off, which may be tied to a lever or a valve thatopens the flow of water.

The external appliance with the flow sensor may be a water consumingdevice such as the programmable (automated) shower 2, the bathtub leveldevice including the drain 4 and faucet 5, the intelligent (automated)toilet 6, the sink faucet (automated sink) 8, the kitchen appliance 11,and the water system 12. Other water consuming devices are possible.

The humidity sensor may detect humidity in the vicinity of theartificial intelligence device 800. For a bathroom, an increase inhumidity indicates that the shower or another water consuming device isin operation and indicates user activity in proximity to the voicecontrolled device.

The pressure sensor may be incorporated in the automated toilet 6 ortoilet seat 7. The pressure sensor generates sensor data that indicateswhether pressure from the user (e.g., seated at the toilet) has beenreceived. The presence of the user indicates that the bathroom is in useand indicates user activity in proximity to the voice controlled device.Similar pressure sensors may be incorporated into flooring or tiles(e.g., versatile tile including lights, speakers, sensors, and/orheaters implemented as flooring). Other pressure sensors may be used tomeasure water flow.

The light sensor may be incorporated into the intelligent mirror (mirrorassembly) 1, the programmable (automated) shower 2, the bathtub sensorydevice 3, the bathtub level device including the drain 4 and faucet 5,the intelligent (automated) toilet 6, the toilet (automated) seat 7, thesink faucet (automated sink) 8, or the light source 9 a and light guides9 b. The change in light in the bathroom indicates that the bathroom isin use and indicates user activity in proximity to the voice controlleddevice.

The inertial sensor may include a gyroscope, an accelerometer, or animpact sensor. The inertial sensor may be included in the artificialintelligence device 800. When the artificial intelligence device 800 ismoved, bumped, turned, or otherwise experiences motion. The output ofthe inertial sensor changes. The controller 804 is configured to monitorthe output of the inertial sensor to detect the change, or compare thevalue of the output of the inertial sensor to a threshold. When theoutput of the inertial sensor changes or exceeds the threshold, thecontroller 804 determines that a user is present or the bathroom is inuse.

Other sensors may directly measure the movement or presence of a user inthe bathroom. A proximity sensor may detect a presence of use in apredetermined distance or within a predetermined distance range. Theproximity sensor may include a microwave or radar sensor. Anotherspecific type of proximity sensor is a gesture sensor that measuresmovements of the user. The controller 804 may identify a gesture from apredetermined said of gestures based on the movement of the user. Thepresence of the user, or a specific gesture from the user, may indicatethat the bathroom is in use and indicates user activity in proximity tothe voice controlled device. The proximity sensors may be included inany of the intelligent bathroom devices or appliances described herein.

Rather than sensor 805, one of the microphones 801 may detect a soundcharacteristic of the water consuming appliances such as a water faucet,a toilet, or a shower. The controller 804 may activate more microphone801 in response to the sound characteristic.

The controller 804 may select a mode in response to the sensor data. Themodes may be defined on the operation of the microphones 801. Thecontroller 804 is configured to operate a first subset of themicrophones 801, or no microphones, in response sensor data describingone type of event and operate a second subset of the microphones 801 inresponse to sensor data describing another type of event. The first typeof event may be the absence of a user in the proximity to the artificialintelligence device 800. The first type of event may trigger a dormantmode. The second type of event may be the presence of a user in theproximity to the artificial intelligence device 800. The second type ofevent may trigger a listening mode.

When the microphones 801 are in the same device such as artificialintelligence device 800, as shown in FIG. 33, the first subset ofmicrophones 801 is at least one microphone in the device and the secondsubset of microphones 801 is a number of microphones greater than thefirst subset of microphones. When the microphones 801 are distributedacross multiple appliances, as shown in FIG. 34, the first subset ofmicrophones 801 may be zero microphones for a particular device, butacross the system is one or more microphones such that at least onemicrophone in the setting (e.g., room or bathroom) is always listeningin the dormant mode. In the listening mode, the second subset ofmicrophones may include all microphones 801 across all devices in thesetting, or multiple microphones in the room.

The first type of event may be detected by sensor 805 (e.g., a flowsensor, a humidity, a pressure sensor, a light sensor, a proximitysensor, or a gesture sensor) has not detected the presence of a userwithin a predetermined amount of time. Examples for the predeterminedamount of time may include 2 minutes, 10 minutes, or 30 minutes. Thesecond type of event may be detected by sensor 805 when the presence ofa user has been detected within the predetermined amount of time.

More microphones (e.g., all microphones) all powered when the secondtype of event has been detected by the sensor 805. Fewer microphones(e.g., 20% of microphones, only 1 microphone, only 2 microphones, oranother quantity of microphones) that is less than all microphones allpowered when the first type of event has been detected by the sensor805.

In addition or in the alternative to microphones, the controller 804 mayselect a full power mode or a low power mode for operation of theartificial intelligence device 800 in response to the sensor data. Thecontroller 204 may select a low power mode for the voice controlleddevice in response to the first type of sensor data and the controllerselects a full power mode for the voice controlled device in response tothe second type of sensor data. In the low power mode, in addition toselected microphones, the artificial intelligence device 800 maydeactivate the speaker and/or wireless communication.

FIG. 35 illustrates another example of the artificial intelligencedevice 800 as a fan assembly 850, including a fan 808 in addition to themicrophone array 801, the controller 804, the communication interface806, and the sensor 805. The fan assembly 850 may also include a heateror a light, which may be steerable as a light guide. Additional,different, or fewer components may be included.

The sensor 805 of the fan assembly 850 may collect sensor data on thequality of air in the bathroom. The sensor 805 may detect oxygen levelsor other chemical levels in the air. The sensor 805 may detect odors.The sensor 805 may detect volatile organic compounds (VOCs) or othercarbon based (organic) chemicals (compounds) that are indicative ofodor.

The sensor 805 of the fan may include any of the motion detectors orgesture sensors described herein to detect motion in proximity of any ofthe appliances of the bathroom. The sensor 805 of the fan may include ahumidity sensor that detects the amount of water in the air. Themicrophone 801 in the fan assembly 850 may collect data indicative ofsounds in the room. The sounds detected by the centrally located fanassembly 850 may include voice commands for any of the other intelligentbathroom devices.

The controller 804 may select a mode for the fan assembly 850 inresponse to the sensor data. In a power saving mode, the controller 804is configured to operate a first subset of the microphones 801 inresponse to the sensor data indicating that no user is present in thebathroom. In a normal mode, the controller 804 is configured to operatea second subset of the microphones 801 in response to sensor datadescribing that a user is present in the bathroom or has been presentfor a predetermined amount of time.

FIG. 36 illustrates a flow chart for operation of the microphone array.The acts of the flow chart may be performed by controller 804 or anothercomponent. Acts may be repeated. Fewer or additional acts may beincluded.

At act S501, the controller 804 provides power supply to components ofone or more bathroom devices and/or of one or multiple voice controlleddevices. The bathroom devices may be additional devices in the vicinityof the voice controlled device. The power supply may be provided bymaintaining closed switches or other circuit elements in circuit pathsincluding each of the components. The components may include microphones801. Other examples for the components include speakers, sensors, orlights. Any combination may be included.

At act S503, the controller 804 initiates a low power mode. The lowpower mode may be applied to the voice controlled device or to at leastone additional devices in the vicinity of the voice controlled device.The lower power mode may be initiated in response to a timer. When apredetermined time has elapsed according to the timer, the lower powermode is initiated.

At act S505, the power supply is removed for at least one of thecomponents in response to the low power mode. For additional devices,the power supply may be removed for all components, all microphones, orthe entire device. For the individual voice controlled device, thecontroller 804 may remove the power supply for one or more components byopening switches or other circuit elements in circuit paths includingeach of the components. For the low power mode, the power supply may beremoved from a specific set of components. The specific set ofcomponents may be designated as non-vital components. The specific setof components may be all components except a single microphone andsupporting circuitry. The specific set of components may be thecontroller 804 and a small number of microphones 801. In the case ofadditional devices

At act S507, the controller 804 receives sensor data from an externalappliance, the sensor data describing an environment of the voicecontrolled device. The sensor data may include any of the examplesdescribed herein. In one example, the external appliance is a toiletseat and the sensor data are pressure data indicative of pressure on thetoilet seat. In another example, the external device is a waterconsuming device and the sensor indicates a water flow of the waterconsuming device. In another example, the external device is a waterfaucet, a toilet, or a shower, and the sensor data includes a soundcharacteristic for water of the water faucet, the toilet, or the shower.In another example, the external device is a fan and the sensor datainclude a sound characteristic, an air characteristic, a humiditycharacteristic, or a light characteristic. In another example, theexternal device is any appliance and the sensor data are generated by aproximity sensor describing a user in proximity to the appliance.

At act S509, the controller 804 initiates a normal power mode inresponse to the sensor data. The normal mode provides power supply tothe components. To initiate the normal power mode, the controller 804closes switches or changes the state of other circuit elements incircuit paths including each of the components.

Remote Charging

FIG. 37 illustrates a remote charging and detection system 900 for thebathroom setting 10 that includes multiple appliances or intelligentbathroom devices that may be connected through at least one home hubcommunication device. The remote charging and detection system 900 maycombined with the fan assembly 850 into a single device. The intelligentbathroom devices may include one or more of an intelligent mirror 1, aprogrammable shower 2, a bathtub sensory device 3, a bathtub leveldevice including a drain 4 and faucet 5, an intelligent toilet 6, atoilet seat 7, a sink faucet 8, and light guides described herein.

The remote charging and detection system 900 generates one or morecharging beams 910 that are configured to scan the bathroom setting 10to identify one or more intelligent bathroom devices. One charging beamor multiple charging beams may be used. The charging beams 910 may scanalong a regular (e.g., grid) pattern in the room, or a random pattern inthe room, to identify intelligent bathroom devices in the path of thescan. In one example, an intelligent bathroom device is identified whenthe charging beam 910 meets a charging receptor. An example chargingreceptor 911 for the mirror 1 is illustrated. Any of the appliances mayinclude a charging receptor.

Once the charging receptor is found the charging beam 910 may remain incontact with the charging receptor for a duration selected by the remotecharging and detection system 900. The duration may be a predeterminedamount of time. The duration may be an amount of time to fully charge abatter of the intelligent bathroom device or charge the battery of theintelligent bathroom device above a threshold level. The battery levelof the intelligent bathroom device may be communicated to the remotecharging and detection system 900 directly or through one of the homehub communication devices described herein. The charging beam 910 maycharge the battery through radio frequency energy that is collected bythe intelligent bathroom device. The charging beam 910 may charge thebattery through a laser beam or other light beam that is collected by aphotovoltaic sensor.

Additional Hardware Architecture

Processor 300 may be a general purpose or specific purpose processor, anapplication specific integrated circuit (ASIC), one or more programmablelogic controllers (PLCs), one or more field programmable gate arrays(FPGAs), a group of processing components, or other suitable processingcomponents. Processor 300 is configured to execute computer code orinstructions stored in memory 352 or received from other computerreadable media (e.g., embedded flash memory, local hard disk storage,local ROM, network storage, a remote server, etc.). The processor 300may be a single device or combinations of devices, such as associatedwith a network, distributed processing, or cloud computing.

Memory 352 may include one or more devices (e.g., memory units, memorydevices, storage devices, etc.) for storing data and/or computer codefor completing and/or facilitating the various processes described inthe present disclosure. Memory 352 may include random access memory(RAM), read-only memory (ROM), hard drive storage, temporary storage,non-volatile memory, flash memory, optical memory, or any other suitablememory for storing software objects and/or computer instructions. Memory352 may include database components, object code components, scriptcomponents, or any other type of information structure for supportingthe various activities and information structures described in thepresent disclosure. Memory 352 may be communicably connected toprocessor 300 via a processing circuit and may include computer code forexecuting (e.g., by processor 300) one or more processes describedherein. For example, memory 298 may include graphics, web pages, HTMLfiles, XML files, script code, shower configuration files, or otherresources for use in generating graphical user interfaces for displayand/or for use in interpreting user interface inputs to make command,control, or communication decisions.

In addition to ingress ports and egress ports, the communicationinterface 353 may include any operable connection. An operableconnection may be one in which signals, physical communications, and/orlogical communications may be sent and/or received. An operableconnection may include a physical interface, an electrical interface,and/or a data interface. The communication interface 353 may beconnected to a network. The network may include wired networks (e.g.,Ethernet), wireless networks, or combinations thereof. The wirelessnetwork may be a cellular telephone network, an 802.11, 802.16, 802.20,or WiMax network, a Bluetooth pairing of devices, or a Bluetooth meshnetwork. Further, the network may be a public network, such as theInternet, a private network, such as an intranet, or combinationsthereof, and may utilize a variety of networking protocols now availableor later developed including, but not limited to TCP/IP based networkingprotocols.

While the computer-readable medium (e.g., memory 352) is shown to be asingle medium, the term “computer-readable medium” includes a singlemedium or multiple media, such as a centralized or distributed database,and/or associated caches and servers that store one or more sets ofinstructions. The term “computer-readable medium” shall also include anymedium that is capable of storing, encoding or carrying a set ofinstructions for execution by a processor or that cause a computersystem to perform any one or more of the methods or operations disclosedherein.

In a particular non-limiting, exemplary embodiment, thecomputer-readable medium can include a solid-state memory such as amemory card or other package that houses one or more non-volatileread-only memories. Further, the computer-readable medium can be arandom access memory or other volatile re-writable memory. Additionally,the computer-readable medium can include a magneto-optical or opticalmedium, such as a disk or tapes or other storage device to capturecarrier wave signals such as a signal communicated over a transmissionmedium. A digital file attachment to an e-mail or other self-containedinformation archive or set of archives may be considered a distributionmedium that is a tangible storage medium. Accordingly, the disclosure isconsidered to include any one or more of a computer-readable medium or adistribution medium and other equivalents and successor media, in whichdata or instructions may be stored. The computer-readable medium may benon-transitory, which includes all tangible computer-readable media.

In an alternative embodiment, dedicated hardware implementations, suchas application specific integrated circuits, programmable logic arraysand other hardware devices, can be constructed to implement one or moreof the methods described herein. Applications that may include theapparatus and systems of various embodiments can broadly include avariety of electronic and computer systems. One or more embodimentsdescribed herein may implement functions using two or more specificinterconnected hardware modules or devices with related control and datasignals that can be communicated between and through the modules, or asportions of an application-specific integrated circuit. Accordingly, thepresent system encompasses software, firmware, and hardwareimplementations.

What is claimed is:
 1. A voice controlled device comprising: a pluralityof microphones configured to receive voice inputs from one or more usersfor operation of the voice controlled device; a communication interfaceconfigured to receive sensor data from an external device, the sensordata describing an environment of the voice controlled device; and acontroller configured to operate a first subset of the plurality ofmicrophones in response to first sensor data and a second subset of theplurality of microphones in response to second sensor data.
 2. The voicecontrolled device of claim 1, wherein the controller selects a low powermode for the voice controlled device in response to the first sensordata and the controller selects a full power mode for the voicecontrolled device in response to the second sensor data.
 3. The voicecontrolled device of claim 1, wherein the controller operates in adormant mode in response to the first sensor data and operates in alistening mode in response to the second sensor data.
 4. The voicecontrolled device of claim 1, wherein the first sensor data indicatesuser activity in proximity to the voice controlled device.
 5. The voicecontrolled device of claim 1, wherein the external device is a waterconsuming device.
 6. The voice controlled device of claim 1, wherein theexternal device is a water faucet, a toilet, or a shower, and the sensordata includes a sound characteristic of the water faucet, the toilet, orthe shower.
 7. The voice controlled device of claim 1, wherein theexternal device is a fan and the sensor data describe humidity in aproximity of the fan.
 8. The voice controlled device of claim 1, whereinthe first sensor data describes a flow of water.
 9. The voice controlleddevice of claim 1, wherein the voice controlled device is associatedwith a first appliance, and the first sensor data describes operation ofa second appliance.
 10. A method for operation of at least one bathroomdevice, the method comprising: providing power to a plurality ofcomponents of the at least one bathroom device; initiating a low powermode for the at least one bathroom device; removing power for at leastone of the plurality of components in response to the low power mode;receiving sensor data from an external device, the sensor datadescribing an environment of the at least one bathroom device;initiating a normal power mode for the at least one bathroom device inresponse to the sensor data; and providing power to the plurality ofcomponents in response to the normal power mode.
 11. The method of claim10, wherein the external appliance is a toilet seat and the sensor dataare pressure data indicative of pressure on the toilet seat or theexternal appliance is a faucet and the sensor data indicate a user inproximity to the faucet.
 12. The method of claim 10, wherein the atleast one bathroom device includes a voice controlled device and atleast one additional device, wherein the low power mode removes powerfrom the at least one additional device and the normal power modeprovides power to the at least one additional device.
 13. The method ofclaim 10, wherein the plurality of components includes a plurality ofmicrophones and the at least one of the plurality of components for thelow power mode is a subset of the plurality of microphones.
 14. Themethod of claim 10, wherein the sensor data indicates user activity inproximity to the at least one bathroom device.
 15. The method of claim10, wherein the external device is a water consuming device.
 16. Themethod of claim 15, wherein the sensor indicates a water flow of thewater consuming device.
 17. The method of claim 10, wherein the externaldevice is a water faucet, a toilet, or a shower, and the sensor dataincludes a sound characteristic for water of the water faucet, thetoilet, or the shower.
 18. The method of claim 10, wherein the externaldevice is a fan and the sensor data include a sound characteristic, anair characteristic, a humidity characteristic, or a lightcharacteristic.
 19. A system of voice controlled devices comprising: aplurality of microphones distributes across multiple voice controlleddevices, the plurality of microphones configured to receive voice inputsfrom one or more users; a sensor configured to collect sensor datadescribing an environment of the voice controlled device; and acontroller configured to operate a first subset of the plurality ofmicrophones in response to a first sensor data and a second subset ofthe plurality of microphones in response to a second sensor data. 20.The system of voice controlled devices of claim 19, wherein the sensoris an accelerometer or a light sensor.